rrdtool advanced topics
play

RRDtool advanced Topics Tobias Oetiker OETIKER+PARTNER AG OpenNMS - PowerPoint PPT Presentation

RRDtool advanced Topics Tobias Oetiker OETIKER+PARTNER AG OpenNMS User Conference 2013 A different kind of Database creating a simple rrd #!/bin/sh 1 PATH =/ scratch/rrd4/bin:$PATH 2 R=rrdtool 3 $R create first.rrd \ 4 --step =300 \


  1. on-disk structure +-------------------------------+ | RRD cookie, DB cfg | Static Header |-------------------------------| : Data Source (DS) Definitions : |-------------------------------| : RR Archive (RRA) Definitions : |===============================| | last update time | Live Head |-------------------------------| : last value for diff : PDP Prep per DS |-------------------------------| : intermediate storage : CDP Prep per RRA and DS |-------------------------------| : RRA Pointers : |===============================| : Round Robin Archives (RRA) : +-------------------------------+

  2. on-disk structure +-------------------------------+ | RRD cookie, DB cfg | Static Header |-------------------------------| : Data Source (DS) Definitions : |-------------------------------| : RR Archive (RRA) Definitions : |===============================| | last update time | Live Head |-------------------------------| : last value for diff : PDP Prep per DS |-------------------------------| : intermediate storage : CDP Prep per RRA and DS |-------------------------------| : RRA Pointers : |===============================| : Round Robin Archives (RRA) : +-------------------------------+

  3. irregular data arrival intervals #!/bin/sh 1 PATH =/ scratch/rrd4/bin:$PATH 2 R=rrdtool 3 $R create real.rrd \ 4 --step =300 \ 5 --start =1199999699 \ 6 DS:distance:COUNTER :600: -40:100 \ 7 RRA:AVERAGE :0.4:1:5 8 9 u(){ 10 $R update real.rrd $1 11 } 12 13 u 1200000000:0 14 u 1200000150:15 15 16 u 1200000310:31 17 u 1200000640:64 18 u 1200000910:91

  4. database after the irregular updates $R fetch real.rrd -s 1200000000 -e 1200000899 AVERAGE 1 distance 1 2 1200000300: 1.0000000000e-01 3 1200000600: 1.0000000000e-01 4 1200000900: 1.0000000000e-01 5 ◮ rrdtool re-binning at work ◮ major difference to a normal db ◮ all bins contain 1.0 ◮ the time is the ’end-time’ of the bin.

  5. database after the irregular updates $R fetch real.rrd -s 1200000000 -e 1200000899 AVERAGE 1 distance 1 2 1200000300: 1.0000000000e-01 3 1200000600: 1.0000000000e-01 4 1200000900: 1.0000000000e-01 5 ◮ rrdtool re-binning at work ◮ major difference to a normal db ◮ all bins contain 1.0 ◮ the time is the ’end-time’ of the bin.

  6. database after the irregular updates $R fetch real.rrd -s 1200000000 -e 1200000899 AVERAGE 1 distance 1 2 1200000300: 1.0000000000e-01 3 1200000600: 1.0000000000e-01 4 1200000900: 1.0000000000e-01 5 ◮ rrdtool re-binning at work ◮ major difference to a normal db ◮ all bins contain 1.0 ◮ the time is the ’end-time’ of the bin.

  7. database after the irregular updates $R fetch real.rrd -s 1200000000 -e 1200000899 AVERAGE 1 distance 1 2 1200000300: 1.0000000000e-01 3 1200000600: 1.0000000000e-01 4 1200000900: 1.0000000000e-01 5 ◮ rrdtool re-binning at work ◮ major difference to a normal db ◮ all bins contain 1.0 ◮ the time is the ’end-time’ of the bin.

  8. optimizing your rrds ◮ update of multi DS RRD is cheep ◮ single update interval per RRD ◮ RRD modification is expensive ◮ RRD size and update performance are independent ◮ RRA complexity affects update performance

  9. optimizing your rrds ◮ update of multi DS RRD is cheep ◮ single update interval per RRD ◮ RRD modification is expensive ◮ RRD size and update performance are independent ◮ RRA complexity affects update performance

  10. optimizing your rrds ◮ update of multi DS RRD is cheep ◮ single update interval per RRD ◮ RRD modification is expensive ◮ RRD size and update performance are independent ◮ RRA complexity affects update performance

  11. optimizing your rrds ◮ update of multi DS RRD is cheep ◮ single update interval per RRD ◮ RRD modification is expensive ◮ RRD size and update performance are independent ◮ RRA complexity affects update performance

  12. optimizing your rrds ◮ update of multi DS RRD is cheep ◮ single update interval per RRD ◮ RRD modification is expensive ◮ RRD size and update performance are independent ◮ RRA complexity affects update performance

  13. fetching data fetch is for reading data from an rrd RRA:AVERAGE :0.5:1:2 \ 1 RRA:AVERAGE :0.5:2:3 2 ◮ one RRA with two 300s entries ◮ one RRA with three 600s entries

  14. fetching data fetch is for reading data from an rrd RRA:AVERAGE :0.5:1:2 \ 1 RRA:AVERAGE :0.5:2:3 2 ◮ one RRA with two 300s entries ◮ one RRA with three 600s entries

  15. playing catch with fetch first pull 300 seconds > rrdtool fetch x.rrd -r 300 \ -s 1200000600 -e 1200000900 AVERAGE 1200000900: 4.0000000000e+01 1200001200: 5.0000000000e+01 then pull 900 seconds > rrdtool fetch x.rrd -r300 \ -s 1200000000 -e 1200000900 AVERAGE 1200000600: 2.5000000000e+01 1200001200: 4.5000000000e+01

  16. fetch recap ◮ looking for complete coverage ◮ resolution is only a suggestion ◮ time stamp in output marks the END of the period ◮ end-time differences caused problems ◮ since 1.3, only the start-time is relevant for coverage ◮ outside the rra everything is NaN

  17. fetch recap ◮ looking for complete coverage ◮ resolution is only a suggestion ◮ time stamp in output marks the END of the period ◮ end-time differences caused problems ◮ since 1.3, only the start-time is relevant for coverage ◮ outside the rra everything is NaN

  18. fetch recap ◮ looking for complete coverage ◮ resolution is only a suggestion ◮ time stamp in output marks the END of the period ◮ end-time differences caused problems ◮ since 1.3, only the start-time is relevant for coverage ◮ outside the rra everything is NaN

  19. fetch recap ◮ looking for complete coverage ◮ resolution is only a suggestion ◮ time stamp in output marks the END of the period ◮ end-time differences caused problems ◮ since 1.3, only the start-time is relevant for coverage ◮ outside the rra everything is NaN

  20. fetch recap ◮ looking for complete coverage ◮ resolution is only a suggestion ◮ time stamp in output marks the END of the period ◮ end-time differences caused problems ◮ since 1.3, only the start-time is relevant for coverage ◮ outside the rra everything is NaN

  21. fetch recap ◮ looking for complete coverage ◮ resolution is only a suggestion ◮ time stamp in output marks the END of the period ◮ end-time differences caused problems ◮ since 1.3, only the start-time is relevant for coverage ◮ outside the rra everything is NaN

  22. Graphing

  23. rrdgraph syntax 101 for graph command syntax, there are two basic rules: 1. --options start with a double dash 2. graphing instructions start with a letter rrdtool graph output DEF:var= rrd : DS : AVARAGE LINE:var# hex-rgb-color :Comment DEF and LINE are graphing instructions .

  24. rrdgraph syntax 101 for graph command syntax, there are two basic rules: 1. --options start with a double dash 2. graphing instructions start with a letter rrdtool graph output DEF:var= rrd : DS : AVARAGE LINE:var# hex-rgb-color :Comment DEF and LINE are graphing instructions .

  25. rrdgraph syntax 101 for graph command syntax, there are two basic rules: 1. --options start with a double dash 2. graphing instructions start with a letter rrdtool graph output DEF:var= rrd : DS : AVARAGE LINE:var# hex-rgb-color :Comment DEF and LINE are graphing instructions .

  26. rrdgraph syntax 101 for graph command syntax, there are two basic rules: 1. --options start with a double dash 2. graphing instructions start with a letter rrdtool graph output DEF:var= rrd : DS : AVARAGE LINE:var# hex-rgb-color :Comment DEF and LINE are graphing instructions .

  27. normal line RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 00:00 02:00 04:00 06:00 DEF:a=x.rrd:a:AVERAGE DEF:b=x.rrd:b:AVERAGE

  28. lower limit RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 --lower-limit= 0

  29. slope mode RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 --slope-mode

  30. anti-anti-aliasing: graph

  31. anti-anti-aliasing: font

  32. line width RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 LINE 1 :b#ff00ff LINE 4 :a#ffaa00

  33. dashed line RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 LINE1:a#ff00ff:: dashes=10,10,80,10 LINE2:b#ffaa00:: dashes=1,3 : dash-offset=10

  34. DEF with :step RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 DEF:a=x.rrd:a:AVERAGE DEF:b=x.rrd:a:AVERAGE: step=1800

  35. DEF with :start RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 DEF:a=x.rrd:a:AVERAGE DEF:b=x.rrd:a:AVERAGE: start=1200011700

  36. DEF with :reduce RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 DEF:b=x.rrd:a:AVERAGE:step=1800: reduce=MIN DEF:c=x.rrd:a:AVERAGE:step=1800: reduce=MAX DEF:a=x.rrd:a:AVERAGE

  37. AREA simple RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 AREA :a#a1003b LINE:b#11a03b

  38. two AREAs RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 AREA :a#a1003b AREA :b#11a03b

  39. transparent AREA RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 AREA:a#a1003b ff AREA:b#11a03b 60

  40. stacked AREA RRDTOOL / TOBI OETIKER 200 180 160 140 120 100 80 60 40 20 0 00:00 02:00 04:00 06:00 AREA:a#a1003b AREA:b#11a03b:...: STACK

  41. time shift RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 CDEF:b=a SHIFT :b:3600

  42. shifting with extra data RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 CDEF:b=a SHIFT :b:3600 DEF:a=x.rrd:a:AVERAGE: start=1199996100

  43. Revers Polish Notation (RPN) Math

  44. RPN basics: Step 0 15 + 23 = 38 1: NAN 2: NAN 3: NAN

  45. RPN basics: Step 1 15 + 23 = 38 [15] 1: 15 2: NAN 3: NAN

  46. RPN basics: Step 2 15 + 23 = 38 [23] 1: 23 2: 15 3: NAN

  47. RPN basics: Step 3 15+23 = 38 [+] 1: 38 2: NAN 3: NAN

  48. math in the graph (+) RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 CDEF :b=a,20,+

  49. simple gradient RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 CDEF:c=a,4,/ AREA:c#77b7ff AREA:c#5aa8ff::STACK AREA:c#2b8fff::STACK AREA:c#0078ff::STACK

  50. the MAX function RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 a b c c=a,b, MAX

  51. the LIMIT function RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 a b b=a,30,70, LIMIT

  52. the TREND function RRDTOOL / TOBI OETIKER 200 180 160 140 120 100 80 60 40 20 0 00:00 02:00 04:00 06:00 a b b=a,3600, TREND

  53. the TREND with early start RRDTOOL / TOBI OETIKER 200 180 160 140 120 100 80 60 40 20 0 00:00 02:00 04:00 06:00 DEF:a=graph-examples.rrd:a:AVERAGE: start=1199996100 a b b=a,3600,TREND

  54. the TREND and SHIFT RRDTOOL / TOBI OETIKER 200 180 160 140 120 100 80 60 40 20 0 00:00 02:00 04:00 06:00 a CDEF:b=a,3600,TREND SHIFT :b:-1800 b

  55. the IF function RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 a b c=a,b, LT ,a,b, IF ,4,-

  56. horizontal gradient RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 a b=a,75,LE,a,75,IF c=a,50,LE,a,50,IF b=a,25,LE,a,25,IF

  57. about invisibility RRDTOOL / TOBI OETIKER 120 110 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 c=a,b,GT,a, UNKN ,IF d=a,b,LT,b, UNKN ,IF

  58. positional drawing count RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 c= COUNT ,3,%,0,EQ,a,UNKN,IF

  59. access the previous value RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 CDEF:c=COUNT,3,%,0,EQ,a,UNKN,IF d=COUNT,3,%,1,EQ, PREV ,c,IF

  60. positional drawing time RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 c= TIME ,1800,%,900,GE,a,UNKN,IF

  61. positional drawing time-shifting RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 c=TIME, 1,- ,1800,%,900,GE,a,UNKN,IF

  62. time and resolution issues RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 c=TIME,1756,%,180,GE,a,UNKN,IF

  63. CDEF internals ◮ data may come in different resolutions ◮ all items in a CDEF must have the same resolution ◮ resolution is expanded to greatest common devisor (gcd) ◮ example: gcd(6,9) = 3, gcd(1,6) = 1 trick: an rrd with one a second step. rrdtool create one.rrd --step=1 DS:one:GAUGE:2:U:U RRA:AVERAGE:0.5:1:1

  64. CDEF internals ◮ data may come in different resolutions ◮ all items in a CDEF must have the same resolution ◮ resolution is expanded to greatest common devisor (gcd) ◮ example: gcd(6,9) = 3, gcd(1,6) = 1 trick: an rrd with one a second step. rrdtool create one.rrd --step=1 DS:one:GAUGE:2:U:U RRA:AVERAGE:0.5:1:1

  65. CDEF internals ◮ data may come in different resolutions ◮ all items in a CDEF must have the same resolution ◮ resolution is expanded to greatest common devisor (gcd) ◮ example: gcd(6,9) = 3, gcd(1,6) = 1 trick: an rrd with one a second step. rrdtool create one.rrd --step=1 DS:one:GAUGE:2:U:U RRA:AVERAGE:0.5:1:1

  66. CDEF internals ◮ data may come in different resolutions ◮ all items in a CDEF must have the same resolution ◮ resolution is expanded to greatest common devisor (gcd) ◮ example: gcd(6,9) = 3, gcd(1,6) = 1 trick: an rrd with one a second step. rrdtool create one.rrd --step=1 DS:one:GAUGE:2:U:U RRA:AVERAGE:0.5:1:1

  67. step=1 trick: high resolution cdef RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 DEF:one=1.rrd:one:AVERAGE c=one,POP,TIME,1756,%,180,GE,a,UNKN,IF

  68. Consolidation functions

  69. finding the average RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 b avg 57.7 VDEF :b=a,AVERAGE GPRINT :b:avg %.1lf

  70. calculating min and max RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 max 97.5 23:40 VDEF:max=a,MAXIMUM min 15.9 02:15 VDEF:min=a,MINIMUM

  71. min max code example LINE:a#456:a VDEF:max=a,MAXIMUM LINE:max#123 VRULE:max#123:maximum GPRINT:max:%.1lf GPRINT:max:%H \ :%M :strftime A VDEF result has a value and a time assigned.

  72. Least Squares Line (y=x*m+b) RRDTOOL / TOBI OETIKER 100 90 80 70 60 50 40 30 20 10 0 00:00 02:00 04:00 06:00 VDEF:slope=a,LSLSLOPE (-0.142) VDEF:int=a,LSLINT (63.4) a lsl=a,POP,COUNT,slope,*,int,+

  73. Holt Winters Aberrant Behaviour Detection

  74. about alert generation ◮ when something unexpected happens send an alert ◮ fixed thresholds are too wide a net ◮ moving averages weigh all data equal ◮ holt winters can predict the future ◮ and no one considers himself clever enough to use it

  75. about alert generation ◮ when something unexpected happens send an alert ◮ fixed thresholds are too wide a net ◮ moving averages weigh all data equal ◮ holt winters can predict the future ◮ and no one considers himself clever enough to use it

  76. about alert generation ◮ when something unexpected happens send an alert ◮ fixed thresholds are too wide a net ◮ moving averages weigh all data equal ◮ holt winters can predict the future ◮ and no one considers himself clever enough to use it

  77. about alert generation ◮ when something unexpected happens send an alert ◮ fixed thresholds are too wide a net ◮ moving averages weigh all data equal ◮ holt winters can predict the future ◮ and no one considers himself clever enough to use it

  78. about alert generation ◮ when something unexpected happens send an alert ◮ fixed thresholds are too wide a net ◮ moving averages weigh all data equal ◮ holt winters can predict the future ◮ and no one considers himself clever enough to use it

  79. rrd - holt winters assumptions ◮ data is periodic in nature ◮ data has continuity ◮ data continuity is periodic ◮ recent data is more important

  80. rrd - holt winters assumptions ◮ data is periodic in nature ◮ data has continuity ◮ data continuity is periodic ◮ recent data is more important

Recommend


More recommend