Metrictank: Building a new time series engine for GrafanaCloud - PowerPoint PPT Presentation

Nov 21, 2023 •336 likes •485 views

Metrictank: Building a new time series engine for GrafanaCloud PERCONA LIVE 2018 Raj Dutt / co-founder Grafana Labs dutt@grafana.com @nopzoraps Grafana everywhere CUSTOMER USER USER CUSTOMER USER CUSTOMER USER The Rise of Beautiful

Metrictank: Building a new time series engine for GrafanaCloud PERCONA LIVE 2018 Raj Dutt / co-founder Grafana Labs dutt@grafana.com @nopzoraps
Grafana everywhere CUSTOMER USER USER CUSTOMER USER CUSTOMER USER
The Rise of Beautiful metric & analytic dashboards Grafana is the leading analytics   software for time series data > 150,000 Signi fi cant   teams actively Enterprise using Grafana, Penetration and growing in the last year How do we make Active and this a sustainable Engaged Global project? Community. 600+ $ Developers
How to monetize? GrafanaCloud A complete Grafana stack. As a service. But what metrics backend?
Grafana is the Switzerland of TSDBs. So why build our own? Why not just use one?
BUT we also needed to meet GrafanaCloud requirements Extremely e ffi cient and reliable - Low resource utilization, margin is important! - High availability - Graphite/whisper No commercial dependencies - Margin, again. Control of our destiny - In fl uxDB Reasonable scaling and maintenance requirement - Low burden on SRE team. - Many Suitable for SaaS deployment and non “cloud native” use cases. - Long term storage. Push/pull. - Prometheus
The Facebook Gorilla Paper… super interesting
Dieter embarks on a crazy experiment - metrictank
Integrate and interoperate. Not replace. No forklift upgrade for users.
  What if we could overcome these limitations? COMPATIBLE WITH BOTH Be long term storage Add tags (dimensionality) Be the centralized HA store Add di ff erent roll-ups Remote write support Graphite-API support PromQL support   Participate in the community Shepherd the community
So what exactly does metrictank do? Read from your choice of inputs: Kafka, Carbon, Prometheus, OpenTSDB, Datadog Compress and save to your choice of backend: Cassandra/ScyllaDB or Google BigTable Performs roll-ups Caches hot data, serves queries Graphite function API   PromQL
Conclusion We didn’t build a TSDB per se. More a purpose built engine. Use whatever makes sense. Interoperability is great. That’s why Grafana supports Graphite, Prometheus, Timescale, ClickHouse, InfluxDB That’s why metrictank supports Cassandra, Bigtable, Graphite, Prometheus Check out metrictank OSS http://www.github.com/grafana/metrictank   But get ready to roll up your sleeves We now have a monetization strategy for the Grafana project Good initial uptake. Check out GrafanaCloud http://www.grafana.com/cloud
THANK YOU

Recommend

Time Series Analysis and Mining with R Time Series Decomposi- tion Time Series Forecasting

R and Time Series Data Time Series Analysis and Mining with R Time Series Decomposi- tion Time Series Forecasting Yanchang Zhao Time Series Clustering Time Series RDataMining.com Classification http://www.rdatamining.com/ R Functions

1.91k views • 42 slides

W HAT IS TIME SERIES D ATA ? W HAT IS TIME SERIES D ATA ? A value over time W HAT IS TIME

T IME S ERIES D ATA P APERS C OVERED Interactive Visualization of Serial Periodic Data John V. Carlis and Joseph A. Konstan Visualizing and Discovering Non-Trivial Patterns in Large Time Series Databases Jessica Lin, Eamonn Keogh,

764 views • 32 slides

Outline Time series and forecasting Time series objects 1 in R Basic time series functionality

Time series and forecasting in R 1 Time series and forecasting in R 2 Outline Time series and forecasting Time series objects 1 in R Basic time series functionality 2 The forecast package 3 Rob J Hyndman Exponential smoothing 4 ARIMA

714 views • 8 slides

Building a scalable time-series database using Postgres Mike Freedman Co-founder / CTO,

Building a scalable time-series database using Postgres Mike Freedman Co-founder / CTO, Timescale mike@timescale.com https://github.com/timescale/timescaledb Time-series data is everywhere, greater volumes than ever before What DB for

689 views • 45 slides

The Quest DRW Series 2014 GM 4500 SERIES 14,200 GVWR 6.0L GAS ENGINE EXAMPLE PICTURE SHOWN

The Quest DRW Series 2014 GM 4500 SERIES 14,200 GVWR 6.0L GAS ENGINE EXAMPLE PICTURE SHOWN EQUIPMENT AND OPTIONS WILL VARY STANDARD FEATURES INCLUDED IN BASE PRICE * Full Length Entry Stepwell Grab Rail * Red Interior Light * Wheel Base

155 views • 4 slides

Introduction to Time Series Basic Concepts Time series concepts well cover Elements of

3/31/2010 Introduction to Time Series Basic Concepts Time series concepts well cover Elements of exploratory time series analysis Motivation, terminology and example Time series plots and classical decomposition Autocovariances

593 views • 26 slides

Time Series Representations for Better Data Mining What can we do with time series data?

Time Series Representations for Better Data Mining What can we do with time series data? Clustering Anomaly (outlier) detection Forecasting What are the problems with time series data? Noise Concept-drift (trend-shift

122 views • 11 slides

Introduction to Time Series Heino Bohn Nielsen 1 of 15 Outline (1) What is a time series? (2)

Econometrics 2 Fall 2005 Introduction to Time Series Heino Bohn Nielsen 1 of 15 Outline (1) What is a time series? (2) Important characteristics of time series data. (3) Data sampling and stochastic processes. Cross-sectional data vs. time

374 views • 8 slides

Why do you care? Time-series data is all over the place. Time-Series Data Kaitlin Duck

Why do you care? Time-series data is all over the place. Time-Series Data Kaitlin Duck Sherwood CS 533c What is Time-Series Data? What is Time-Series Data? Lines. Usually periodic Source: van Wijk and van Selow, Cluster and

85 views • 4 slides

Building a real-time recommendation engine with Neo4j OSCON 2017 William Lyon @lyonwj William

Building a real-time recommendation engine with Neo4j OSCON 2017 William Lyon @lyonwj William Lyon Developer Relations Engineer @neo4j will@neo4j.com @lyonwj lyonwj.com Agenda Use cases Recommender systems

805 views • 51 slides

What the heck is time-series data (and why do I need a time-series database?) Ajay Kulkarni |

What the heck is time-series data (and why do I need a time-series database?) Ajay Kulkarni | Co-founder/CEO | ajay@timescale.com Fastest growing database category Source: DB Engines In this talk 1. What is time-series data? (hint: its

540 views • 36 slides

Working w ith more than one time series VISU AL IZIN G TIME SE R IE S DATA IN P YTH ON Thomas

Working w ith more than one time series VISU AL IZIN G TIME SE R IE S DATA IN P YTH ON Thomas Vincent Head of Data Science , Ge y Images Working w ith m u ltiple time series An isolated time series A le w ith m u ltiple time series

505 views • 29 slides

Time Series A time series is a collection of data obtained by observing a response variable at

ST 430/514 Introduction to Regression Analysis/Statistics for Management and the Social Sciences II Time Series A time series is a collection of data obtained by observing a response variable at various times. Examples The monthly Case-Shiller

351 views • 8 slides

ClickHouse for Time-Series Alexander Zaitsev Agenda What is special about time series What is

ClickHouse for Time-Series Alexander Zaitsev Agenda What is special about time series What is ClickHouse How ClickHouse can be used for time series Altinity Background Premier provider of software and services for ClickHouse

897 views • 55 slides

Lecture 6 Discrete Time Series 9/21/2018 1 Discrete Time Series Stationary Processes A

Lecture 6 Discrete Time Series 9/21/2018 1 Discrete Time Series Stationary Processes A stocastic process (i.e. a time series) is considered to be strictly stationary if the properties of the process are not changed by a shift in origin. In

760 views • 33 slides

Introd u ction to the Co u rse TIME SE R IE S AN ALYSIS IN P YTH ON Rob Reider Adj u nct

Introd u ction to the Co u rse TIME SE R IE S AN ALYSIS IN P YTH ON Rob Reider Adj u nct Professor , NYU - Co u rant Cons u ltant , Q u antopian E x ample of Time Series : Google Trends TIME SERIES ANALYSIS IN PYTHON E x ample of Time Series :

407 views • 36 slides

Dynamic Time Warping Averaging of Time Series allows Faster and more Accurate Classification F.

Dynamic Time Warping Averaging of Time Series allows Faster and more Accurate Classification F. Petitjean G. Forestier G.I. Webb A.E. Nicholson Y. Chen E. Keogh Compute average The Ubiquity of Time Series Sensors on machines Stock prices Wearables

638 views • 30 slides

Time series Decomposing a series into meaningful components R.W. Oldford Time series data -

Time series Decomposing a series into meaningful components R.W. Oldford Time series data - Sunspot activity Recall the measurements of monthly mean relative sunspot numbers from 1749 to 1983 plotted as Monthly sunspot activity 200 Sunspot

860 views • 71 slides

Time Series Data Visualization Class 2, Part A 2 1 Time Series Data Fundamental

Large Scale Information Visualization Jing Yang Fall 2007 1 Time Series Data Visualization Class 2, Part A 2 1 Time Series Data Fundamental chronological component to the data set Random sample of 4000 graphics from 15 of

763 views • 31 slides

Merging time series MAN IP U L ATIN G TIME SE R IE S DATA W ITH XTS AN D ZOO IN R Je re y

Merging time series MAN IP U L ATIN G TIME SE R IE S DATA W ITH XTS AN D ZOO IN R Je re y R y an Creator of x ts and q u antmod Introd u cing merge () Combine series b y col u mn cbind() and merge() Database st y le joins on inde x ( i . e

597 views • 18 slides

Lecture 6 Discrete Time Series Colin Rundel 02/06/2017 1 Discrete Time Series 2 Stationary

Lecture 6 Discrete Time Series Colin Rundel 02/06/2017 1 Discrete Time Series 2 Stationary Processes A stocastic process (i.e. a time series) is considered to be strictly stationary if the properties of the process are not changed by a shift

498 views • 30 slides

Time Series Data Visualization 2 1 Time Series Data Fundamental chronological component to

Information Visualization Jing Yang Spring 2007 1 Time Series Data Visualization 2 1 Time Series Data Fundamental chronological component to the data set Random sample of 4000 graphics from 15 of worlds newspapers and magazines

315 views • 31 slides

Timeseries kinds and applications MAC H IN E L E AR N IN G FOR TIME SE R IE S DATA IN P YTH

Timeseries kinds and applications MAC H IN E L E AR N IN G FOR TIME SE R IE S DATA IN P YTH ON Chris Holdgraf Fello w, Berkele y Instit u te for Data Science Time Series MACHINE LEARNING FOR TIME SERIES DATA IN PYTHON Time Series

802 views • 37 slides

Quality of Similarity Rankings in Time Series T. Bernecker, in Time Series M. E. Houle, H.-P.

Quality of Similarity Rankings Quality of Similarity Rankings in Time Series T. Bernecker, in Time Series M. E. Houle, H.-P. Kriegel, P. Krger, 12th International Symposium on Spatial and Temporal M. Renz, E. Schubert, Databases (SSTD

1.14k views • 18 slides