Write your own monitoring About RRDtool Programming software with - PowerPoint PPT Presentation

Write your own monitoring software with RRDtool Tobias Oetiker Motivation Write your own monitoring About RRDtool Programming software with RRDtool Summary Tobias Oetiker ISG.EE - ETH Zürich OSCON 2005 Portland, Oregon, August 3, 2005

Write your own Netflow Data presented with monitoring software with RRDtool Fluxscope Tobias Oetiker Motivation About RRDtool Programming Summary

Content Write your own monitoring software with RRDtool Tobias Oetiker Motivation Motivation The world today About data collection The world today A graph says more than 1’000 words About data collection About RRDtool A graph says more than Programming Summary 1’000 words About RRDtool Programming Summary

Write your own Numbers everywhere monitoring software with RRDtool Tobias Oetiker Motivation The world today About data collection A graph says more than 1’000 words About RRDtool ◮ 15 users online Programming ◮ 1.34 PB free tape space Summary ◮ 31’223 ifInOctets ◮ 100 degrees in the server room ◮ 87 Mb/s transfer rate

Write your own All these questions monitoring software with RRDtool Tobias Oetiker Motivation The world today ◮ Are we running out of About data collection A graph says more than 1’000 words diskspace? About RRDtool ◮ Is there a pattern in the high Programming server load? Summary ◮ Is there really global warming? ◮ Can we do something about it? ◮ Collect that data!

How to collect data Write your own monitoring software with RRDtool Tobias Oetiker Motivation The world today About data collection A graph says more than 1’000 words About RRDtool ◮ collect everything, analyze Programming later (aka never) Summary ◮ data with known properties makes life simpler ◮ from research to production

Write your own Requirements for a solution monitoring software with RRDtool Tobias Oetiker Motivation The world today About data collection A graph says more than 1’000 words About RRDtool ◮ handle counters and gauges Programming Summary ◮ be nice to the server ◮ setup for specific task ◮ data-pre processing ◮ and forget about it ...

Condor use at UW Write your own monitoring software with RRDtool Tobias Oetiker Motivation The world today About data collection A graph says more than 1’000 words About RRDtool Programming Summary

Write your own Making data accessible monitoring software with RRDtool Tobias Oetiker Motivation The world today About data collection A graph says more than ◮ graphical 1’000 words About RRDtool representation Programming ◮ eye candy for the Summary people ◮ analysis at a glance ◮ make my boss look good to his boss

Data post-processing Write your own monitoring software with RRDtool Tobias Oetiker Motivation The world today About data collection A graph says more than 1’000 words About RRDtool Programming ◮ highlight interesting Summary properties ◮ combine data from several sources ◮ add extra Information

Content Write your own monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool Motivation History Features Graphing features Programming About RRDtool Summary History Features Graphing features Programming Summary

Write your own 1995: MRTG was only a start monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool History Features Graphing features ◮ logfiles in text format Programming Summary ◮ gnuplot for graphs ◮ 1996 - MRTG is used for amazing tasks ◮ more performance ◮ more flexibility

The rrd TOOL Write your own monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool History Features ◮ a building block Graphing features Programming ◮ basis for a better Summary MRTG (2nd system) ◮ basis for hundreds of other tools ◮ Google says: “industry standard”

The Round Robin Database Write your own monitoring software with RRDtool Tobias Oetiker Motivation ◮ lossy storage, fixed file size About RRDtool History ◮ current data is the most Features Graphing features interesting Programming ◮ consolidation functions for Summary feature extraction ◮ artificial data-sources ◮ holt-winters aberrant behavior detection ◮ constant step size ◮ fixed disk space

Write your own Feeding Data monitoring software with RRDtool Tobias Oetiker Motivation ◮ combat jitter by taking About RRDtool History acquisition time into Features Graphing features account. Programming ◮ preserve data-volume Summary ◮ on-the-fly data validation ◮ deal with unknown data ◮ time is a one way street

Write your own Basic Graphing monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool History ◮ auto scaling Features Graphing features ◮ auto labeling Programming Summary ◮ sensible defaults ◮ quick results ◮ anti-aliased output ◮ multiple formats (png/eps/pdf/svg)

Write your own Advanced Graphing monitoring software with RRDtool Tobias Oetiker ◮ change colors, fonts, Motivation sizes About RRDtool History ◮ data from several Features Graphing features databases Programming Summary ◮ data processing with RPN math ◮ alpha transparency (rrggbbaa) ◮ support of locale based character encoding ◮ truetype fonts

Content Write your own monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool Motivation Programming Language bindings Basic operations About RRDtool Advanced Graphing Summary Programming Language bindings Basic operations Advanced Graphing Summary

On the command line Write your own monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Just another Unix command. Language bindings Basic operations code Advanced Graphing Summary rrdtool cmd arg example rrdtool info demo.rrd

Via STDIN Write your own monitoring software with RRDtool Tobias Oetiker Motivation Save startup time by feeding several commands. About RRDtool code Programming Language bindings echo cmd arg | rrdtool - Basic operations Advanced Graphing example Summary echo info demo.rrd | rrdtool - ◮ several commands ◮ fast operation ◮ simple interface

Write your own perl module monitoring software with RRDtool Tobias Oetiker Motivation I love coding in perl ... About RRDtool Programming code Language bindings Basic operations use RRDs; Advanced Graphing my $ return = RRDs:: cmd arg ; Summary example use RRDs; use Data::Dumper my $ret = RRDs::info "demo.rrd"; print Dumper $ret;

Write your own Other Bindings monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Language bindings ◮ RRDcgi Basic operations Advanced Graphing ◮ Perl Pipes - RRDp Summary ◮ Python ◮ Tcl ◮ and more externally

Write your own Creating Round Robin Databases monitoring software with RRDtool Tobias Oetiker ◮ what data do I want to look at? Motivation ◮ how long do I want to keep it? About RRDtool Programming ◮ what are the interesting properties of my data? Language bindings Basic operations ◮ what to keep in one rrd file? Advanced Graphing Summary Command rrdtool create filename [ –start start time ] [ –step step ] [ DS: ds-name : DST : heartbeat : min : max ] [ RRA: CF : xff : steps : rows ] DST : COUNTER, GAUGE, DERIVE, ABSOLUTE, New in 1.2: COMPUTE CF : AVERAGE, MIN, MAX, LAST, New in 1.2: Aberrant Behavior Detection .

Write your own Create Example monitoring software with RRDtool Tobias Oetiker example Motivation rrdtool create example.rrd --step=60\ About RRDtool DS:in:COUNTER:600:0:1000 \ Programming Language bindings RRA:AVERAGE:0.5:1:100 \ Basic operations Advanced Graphing RRA:AVERAGE:0.5:10:100 \ Summary RRA:MAX:0.5:10:100 ◮ resolution 60 seconds ◮ COUNTER data source ◮ updates at least every 600 seconds ◮ accept rates between 0 and 1000 ◮ three archive with 100 entries ◮ store unknown unless half are known.

Write your own Adding Data into a Round Robin monitoring software with RRDtool Database Tobias Oetiker Motivation About RRDtool Programming Command Language bindings Basic operations rrdtool update filename [ –template ds:ds:... ] Advanced Graphing Summary time : value : value ...[...] ◮ data value and acquisition time ◮ timestamps have to increase with every update ◮ instead of time use N for now ◮ templates are for redundancy only

Write your own Update Example monitoring software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Language bindings Basic operations Advanced Graphing example Summary rrdtool update example.rrd \ --template in N:39344

Write your own Data re-bin-ing monitoring software with RRDtool Tobias Oetiker Motivation 700 octets/sec About RRDtool 1 2 Programming 280k Language bindings octets Basic operations Advanced Graphing Summary 400s 3 300s 300s Data Aquisition Data Storage 300s 300s

Data Consolidation Write your own monitoring software with RRDtool Tobias Oetiker RRA with 1 PDP Motivation About RRDtool Programming Language bindings Basic operations Advanced Graphing Summary AVERAGE RRA with 2 PDP AVERAGE RRA with 3 PDP