I’ve recently built a system for monitoring network traffic using the MRTG, RRDTool and Routers2 systems. While learning how to do this I’ve read a fair amount of text however the majority of the information online is either all about MRTG or RRDtool or Routers2.cgi so I couldn’t find a guide going over all three. The following is a guide I’ve created in the process so another sysadmin can save perhaps a spot of time. Please let me know of any corrections, this has worked for me but please remember as with anything you find online, your mileage may vary.

You can see a working example of what the finished system will look like…  http://www.steveshipway.org/cgi-bin/routers2.pl

So to kick off build default centos 6.3 server and install the following features;

Apache with PHP support, MySQL Server and ntp, if you have done a minimal install install perl-cgi.

yum install apache2 php mysql-server ntpd perl-CGI.x86_64 -y

The time is very important on a monitoring server so this host will be synchronized with some regional NTP servers as follows;

chkconfig ntpd on

Edit your /etc/ntp.conf and add whatever servers you wish to use. Im in the UK so these are fine. I’ve added iburst after each server to speed up synchronisation between the hosts so it’s faster getting the most accurate time.

# Use public servers from the pool.ntp.org project.

# Please consider joining the pool (http://www.pool.ntp.org/join.html).

server 0.uk.pool.ntp.org iburst

server 1.uk.pool.ntp.org iburst

server 2.uk.pool.ntp.org iburst

Start your ntp service /etc/init.d/ntpd start, check for errors.

Configure your system with a static IP and switch off selinux if using it. You can leave this on if you wish however this is an internal host so I never bother. You can disable iptables if you wish, I like to keep it running. Remember the server monitors the hosts by querying them so as long as your outbound connections are not impeded and you have inbound port 80 for the routers2 web page allowed (restricted to some subnets) then that is all the firewall rules you will require.

The next step is to create a user for MRTG.

useradd -c “MRTG Service account” mrtg

passwd ****

Now to start the install of MRTG, for this we need MRTG and also RRDTool and RRDTools perl module. We will also need Perl_GD for drawing all the pretty pictures.

yum install mrtg rrdtool rrdtool-perl perl-GD -y

MRTG can work completely independently of RRDTool. MRTG makes lots of nice graphs for you to view with an accompanying web page for whatever metric you choose to measure. RRDTool acts as a storage repository that makes MRTG so much more efficient with the data it would normally save in lots of .log files. This means once you’re saving in .rrd format MRTG will no longer populate .log files or create web pages for you, at all. Later we’ll install routers2.cgi, this takes care of the web page in a very user friendly format.

Once yum has installed MRTG the default path for the MRTG site files /var/www/mrtg/

This is now the point when you can test MRTG functionality. Ensure that apache is running /etc/init.d/httpd [status][start]

For this test you will need a device that is running snmp version 1,2 or 3. We will use a program included with MRTG called cfgmaker. You could write your MRTG config files by hand however this would be madness and cfgmaker is a great tool that will take care of the grunt work. cfgmaker has a large number of options that can be set with it and I wont go into them all as there are pages dedicated to this on the internet, so I’ve included the ones that I feel are most suited to the network graphing platform I’m building. See http://oss.oetiker.ch/mrtg/doc/cfgmaker.en.html for more information.

If you notice the final section of the cfgmaker command public@$hostname, this is the snmp community string and the host you wish to monitor. I’ve included a global config that looks for the hostname via dns however you can add the IP address alone if you wish, simply remove this line. This final option in cfgmaker is to specify the location and name of the new config file you will create.

Normal MRTG appends to one central config file the mrtg.cfg and just about everything you read on the internet doesn’t deviate from this. When you use routers2.cgi it prefers your cfg files separate for each host so a new .cfg will be required for each device.

*With DNS lookup*

cfgmaker –global “workdir: /var/www/mrtg/” –dns-domain=somedomain.co.uk –global “Options[_]: growright,bits” –no-down –ifref=name –ifdesc=descr public@$hostname –output /etc/mrtg/$hostname.cfg

Once you have created your MRTG config file you need to have the MRTG executable parse the file. Load the mrtg config into the location of your mrtg executable.

/usr/bin/mrtg /etc/mrtg/$host.cfg

When you run this for the first time you will get the following warning;

———————————————————————–

 

ERROR: Mrtg will most likely not work properly when the environment

variable LANG is set to UTF-8. Please run mrtg in an environment

where this is not the case. Try the following command to start:

 

env LANG=C /usr/bin/mrtg /etc/mrtg/$hostname.cfg

———————————————————————–

Re-invoke the same command as it says with the environment variable set to LANG=C. We’ll script this into our cron job later but for now this is fine.

Run it once;

env LANG=C /usr/bin/mrtg /etc/mrtg/BHOST.cfg

Then again (you only need to do this for the first time);

env LANG=C /usr/bin/mrtg /etc/mrtg/BHOST.cfg

You can then view your MRTG output by creating an overall html page for each device.

indexmaker –output mainindex.htm /etc/mrtg/$host.cfg

At this point I’ll save you a bit of time. Chances are you will be doing this on a remote test server like myself. Rightly so the /mrtg config file has been restricted to localhost, to fix this edit /etc/httpd/conf.d/mrtg.cfg

This is on my internal network with no access to the outside so I’ve set the defaults to;

Alias /mrtg /var/www/mrtg

 

<Location /mrtg>

Order allow,deny

Allow from all

Allow from 127.0.0.1

Allow from ::1

# Allow from .example.com

</Location>

Check your web browser and you will see your chosen hosts traffic metrics $server/mrtg/mainindex.htm

That is now the base MRTG up and running.

Adding RRDTool.

We’ll move on to using RRDTool to get all that data placed into rrd files. There are two bits of information you need to include in your cfgmaker command as noted http://oss.oetiker.ch/mrtg/doc/mrtg-rrd.en.html

The path to your rrdtool executable [PathAdd:], and the path to your rrd perl module [LibAdd:]. These will be different depending on how you installed rrd.

Include these as global options in your cfg maker command. I’ve also included the fork option will create multiple child instances of mrtg while polling for snmp data and return the answer to the parent process. 10 seems as good a number as any on a dedicated box.

cfgmaker –global “workdir: /var/www/mrtg/” –dns-domain=somedomain.co.uk –global “Options[_]: growright,bits” –global “LogFormat: rrdtool” –global “PathAdd: /usr/bin” –global “LibAdd: /usr/lib64/perl5″ –no-down –ifref=name –ifdesc=descr –global “Forks: 10″ public@$hostname –output /etc/mrtg/$hostname.cfg

Run your configmaker command and follow by running; env LANG=C /usr/bin/mrtg /etc/mrtg/$host.cfg.

Eh volia, you will know it’s working due to your lack of log files and lack of html pages in your workdir and or course the presence of some .rrd files.

Viewing our MRTG and RRD data with Routers2.cgi

Now for our web interface. We’ll be using Routers2.cgi.

First of all you’ll need to wget the latest version from http://www.steveshipway.org/software/index2.html

Untar it and proceed to run through the ./install script making sure the locations of files requested are correct.

Once the script has finished have a quick check in your httpd.conf to see if mod_expires is loaded, if it’s commented out uncomment it and restart httpd/apache service.

If you head over to $server/cgi-bin/routers2.cgi you’ll see a nice web interface. What we now need to do is generate a config file for each device that you wish to monitor. It can be hard work knocking up a cfgmaker command each time for people that don’t know what it does so the best way I found is to generate a bash script and add the devices that way. The following… or thereabout will do just fine…

#!/bin/bash

 

echo -n “What is the name of the device you wish to monitor, i.e. core-switch-a1 followed by [enter]? ”

read device

echo “You have selected “$device

echo -n “What is the snmp community string? [enter]? ”

read community

echo “The snmp community is “$community

 

cfgmaker –global “workdir: /var/www/mrtg/” –dns-domain=somedomain.co.uk –global “Options[_]: growright,bits” –global “LogFormat: rrdtool” –global “PathAdd: /usr/bin” –global “LibAdd: /usr/lib64/perl5″ –no-down –ifref=name –ifdesc=descr –global “Forks: 10″ $community@$device –output /etc/mrtg/$device.cfg

 

Remember that this version of this script will look up the hosts DNS name, you might not want to do this but I do because if it fails it bombs out and that means whoever has ran the command has more than likely entered the host name wrong, where as if you add the IP alone it will add the incorrect device and collect no data.

Once you’ve added all of your devices the last thing is to get them checked at regular intervals. The default is five minutes however I think a one minute check is suitable in the on-demand world of today. There are two ways to have mrtg processing your devices, you can run the service as a deamon or you can set up a cron job. I was instantly attracted to configuring it as a deamon however if the deamon fails I don’t fancy getting up in the middle of the night to start it or worse checking systems monitor in the morning and finding I’ve collected no data for eight hours. I decided crontab was the way to go, however I’ll show you how to do both.

Deamon;

To configure the look up as a deamon you have to ensure you add an extra global config command to each .cfg file so this will have to be added into your script. The extra command is;

–global ‘RunAsDaemon: Yes’

After that ensure the MRTG deamon is running on init with chkconfig mrtg on. For Debian people I’m guessing you will add it to /etc/rd.local or thereabouts.

Crontab;

The seconds option is to use crontab, edit crontab with ‘crontab -e’ and add;

*/1 * * * * env LANG=C /usr/bin/mrtg /etc/mrtg/*.cfg –logging /var/log/mrtg.log

This includes the C language environment variable and it’ll check it every minute. I’m afraid there are no prizes here for figuring how to increase or decrease the interval of checks.

…and that’s that. One thing I’ve not really went into is security, I’d set up my MRTG user to run this for other users as you wouldn’t want to run it as root. You could set the mrtg user to start with a bash -r shell and only allow access to this script, that should lock it down quite significantly.

That’s all you need. All devices you add should be monitored every minute for network traffic. Any questions, corrections or comments please let me know.