Gaps in Cacti Graphs

Posted by
Facebooktwitterredditpinterestlinkedintumblr

Issue: Unexpectedly getting gaps in Cacti graphs that resulted in false alarm in NOC network weathermap. Having gaps isn’t good as visual data is lost, for reporting this gives wrong information and lastly entails an issue somewhere in Cacti.

Cacti gaps in graphs


network weathermap false alarm

Platform: Cacti all software versions.

Solution: The gaps observed in Cacti graphs was only happening on one graph which was very odd. I started to look at device port traffic statistics in question. Port statistics (Input and output queue) were increasing.

hostname-device#sh  int g0/21 | i input rate|output rate
  5 minute input rate 101305000 bits/sec, 20562 packets/sec
  5 minute output rate 59618000bits/sec, 20609 packets/sec

Secondly, did an SNMP walk from Cacti to the device and was getting correct data.

[email protected]:~# snmpwalk -v2c -cMYKEY   a.b.c.d .1.3.6.1.2.1.31.1.1.1.6.10121
IF-MIB::ifHCInOctets.10121 = Counter64: 51916403857963

Thirdly, I checked the RRD file. It contained NaN values. I’ve noticed that the max value for both data source traffic_in and traffic_out were set to 1.0000000000e+08 (that is 100Mbps). The values that were retrieved by Cacti via SNMP query were larger than 1.0000000000e+08, hence the gaps in Cacti graphs. Issue was fixed after I modified the data source max value on both traffic_in and traffic_out to 1.0000000000e+15.

[email protected]:/opt/cacti/rra/7# rrdtool info 36.rrd
filename = "36.rrd"
rrd_version = "0003"
step = 60
last_update = 1409760361
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+08
ds[traffic_in].last_ds = "4673633057231"
ds[traffic_in].value = 2.8749866667e+05
ds[traffic_in].unknown_sec = 0
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+08
ds[traffic_out].last_ds = "2020598186851"
ds[traffic_out].value = 1.1551450000e+04
ds[traffic_out].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 500
rra[0].cur_row = 447
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 600
rra[1].cur_row = 549
rra[1].pdp_per_row = 1
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[1].cdp_prep[1].value = NaN
rra[1].cdp_prep[1].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 700
rra[2].cur_row = 398
rra[2].pdp_per_row = 6
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 0.0000000000e+00
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[2].cdp_prep[1].value = 0.0000000000e+00
rra[2].cdp_prep[1].unknown_datapoints = 0

[email protected]:/opt/cacti/rra/7# rrdtool tune 36.rrd --maximum traffic_in:1.0000000000e+15
[email protected]:/opt/cacti/rra/7# rrdtool tune 36.rrd --maximum traffic_out:1.0000000000e+15
[email protected]:/opt/cacti/rra/7# rrdtool info 36.rrd
filename = "36.rrd"
rrd_version = "0003"
step = 60
last_update = 1409761081
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 120
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+15
ds[traffic_in].last_ds = "51919274509420"
ds[traffic_in].value = 8.0730327119e+05
ds[traffic_in].unknown_sec = 0
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 120
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+07
ds[traffic_out].last_ds = "48365174072129"
ds[traffic_out].value = 5.2109589831e+05
ds[traffic_out].unknown_sec = 0

Mystery solved and Cacti graph is now in good shape.

Cacti gaps in graphs fixed

Leave a Reply

Your email address will not be published. Required fields are marked *

*