There are many tutorials on setting up collectd as an agent on a machine. However, I have not found
many tutorials that describe how to setup a centralized "collectd" server with multiple collectd agents. This setup will
aggregate system metrics from multiple clients and sends them to Graphite.
This is my attempt to note down the steps that I used for setting up centralized collectd metric collection system.
Download latest version of collectd.tar.gz (collectd-5.5 or more). Collectd, by default,contains many plugins e.g. cpu,load,disk,graphite(write_graphite),redis, mysql etc. As a result, it is possible to capture almost all the system metrics.
Basic installation of collectd involves usual steps:- ./configure and make install for collectd server as well as collectd agent:
# tar -xvf collectd-version.tar.gz
# cd collectd-version
#./configure --disable-turbostat # if SL <6.6 or CentOS < 6.6
# make
# make install
Copy collectd daemon file to /etc/init.d
# cp ./contrib/redhat/init.d-collectd /etc/init.d/collectd
# chmod +x /etc/init.d/collectd
Make soft links for binaries in /opt/collectd/bin
# ln-s /opt/collectd/sbin/collectdmon /usr/bin/collectdmon
# ln -s /opt/collectd/sbin/collectd /usr/bin/collectd
If you are sending collectd metrics to graphite, please make sure that graphite is installed and is running prior to compiling collectd. It is presumed that graphite and collectd server are on the same machine in this use-case.
Setting up collectd server
Once collectd is installed as described above, modify the /opt/collectd/etc/collectd.conf file to contain the following:
Hostname "hostname"
FQDNLookup true
BaseDir "/opt/collectd/var/lib/collectd"
PIDFile "/opt/collectd/var/run/collectd.pid"
PluginDir "/opt/collectd/lib/collectd"
TypesDB "/opt/collectd/share/collectd/types.db"
Interval 10
LoadPlugin logfile
<Plugin logfile>
LogLevel info
File "/var/log/collectd.log"
Timestamp true
PrintSeverity true
</Plugin>
LoadPlugin network
<Plugin network>
Listen "*" "25826"
</Plugin>
LoadPlugin interface
<Plugin interface>
Interface "eth0"
</Plugin>
LoadPlugin write_graphite
<Plugin write_graphite>
<Node "graphing">
Host "localhost"
Port "2003"
Protocol "tcp"
LogSendErrors true
Prefix "collectd."
StoreRates true
AlwaysAppendDS false
EscapeCharacter "_"
</Node>
</Plugin>
Make adjustments for your network as needed.
Run collectd using:
# service collectd start
Some useful information that is required while setting up:
Default directories for collectd:
Plugins -/opt/collectd/lib/collectd
Binaries - /opt/collectd/sbin/collectd
Configuration file: /opt/collectd/etc/collectd.conf
Run collectd like this:
/opt/collectd/sbin/collectd -C /opt/collectd/etc/collectd.conf
Test Collectd configuration:
#collectd -t
Test Collectd plugin configuration:
#collectd -T
Check netstat output:
# nestat -naptul |grep "25826"
Setting up Collectd agent
Now, we are going install the collectd agent on the client machine and then tell it to send the metrics to the collectd server (not Graphite). The collectd clients do not need "write_graphite" plugin and can use the older version of Collectd rpms that are available in CentOS/SL repositories. So, on each client, run:
# yum install collectd collectd-utils
Modify /etc/collectd.conf config file as per your requirement:
Hostname "hostname"
FQDNLookup true
BaseDir "/var/lib/collectd"
PIDFile "/var/run/collectd.pid"
PluginDir "/usr/lib/collectd"
TypesDB "/usr/share/collectd/types.db"
Interval 10
#Timeout 5
ReadThreads 5
LoadPlugin logfile
<Plugin logfile>
LogLevel info
File "/var/log/collectd.log"
Timestamp true
PrintSeverity true
</Plugin>
LoadPlugin network
<Plugin network>
Server "collectd-server.domain.com" "25826"
</Plugin>
LoadPlugin cpu
LoadPlugin load
LoadPlugin disk
LoadPlugin memory
LoadPlugin processes
Include "/etc/collectd/filters.conf"
Include "/etc/collectd/thresholds.conf"
Be sure to configure the network plugin with your collectd server information.
With this configuration, client metrics statistics are sent to collectd server on port 25826. These are further sent to Graphite. If you want to spice up web front-end, you can use grafana and show the trend of system metrics.
Enabling python plugin
If you wish to enable python and iptables plugin support, please do the following:
# yum install python-devel
# yum install iptables-devel
Now, re-compile the collectd source package once again for these modules.
# cd collectd-version
# ./configure --enable-python --enable-iptables
# make
# make install
This process is same for any additional plugins that you may wish to add - e.g. mysql, postgres etc.
Please check the Modules output carefully for the plugin support while configuring source package during compilation.
Some of the useful links that I encountered while setup:
This is my attempt to note down the steps that I used for setting up centralized collectd metric collection system.
Download latest version of collectd.tar.gz (collectd-5.5 or more). Collectd, by default,contains many plugins e.g. cpu,load,disk,graphite(write_graphite),redis, mysql etc. As a result, it is possible to capture almost all the system metrics.
Basic installation of collectd involves usual steps:- ./configure and make install for collectd server as well as collectd agent:
# tar -xvf collectd-version.tar.gz
# cd collectd-version
#./configure --disable-turbostat # if SL <6.6 or CentOS < 6.6
# make
# make install
Copy collectd daemon file to /etc/init.d
# cp ./contrib/redhat/init.d-collectd /etc/init.d/collectd
# chmod +x /etc/init.d/collectd
Make soft links for binaries in /opt/collectd/bin
# ln-s /opt/collectd/sbin/collectdmon /usr/bin/collectdmon
# ln -s /opt/collectd/sbin/collectd /usr/bin/collectd
If you are sending collectd metrics to graphite, please make sure that graphite is installed and is running prior to compiling collectd. It is presumed that graphite and collectd server are on the same machine in this use-case.
Setting up collectd server
Once collectd is installed as described above, modify the /opt/collectd/etc/collectd.conf file to contain the following:
Hostname "hostname"
FQDNLookup true
BaseDir "/opt/collectd/var/lib/collectd"
PIDFile "/opt/collectd/var/run/collectd.pid"
PluginDir "/opt/collectd/lib/collectd"
TypesDB "/opt/collectd/share/collectd/types.db"
Interval 10
LoadPlugin logfile
<Plugin logfile>
LogLevel info
File "/var/log/collectd.log"
Timestamp true
PrintSeverity true
</Plugin>
LoadPlugin network
<Plugin network>
Listen "*" "25826"
</Plugin>
LoadPlugin interface
<Plugin interface>
Interface "eth0"
</Plugin>
LoadPlugin write_graphite
<Plugin write_graphite>
<Node "graphing">
Host "localhost"
Port "2003"
Protocol "tcp"
LogSendErrors true
Prefix "collectd."
StoreRates true
AlwaysAppendDS false
EscapeCharacter "_"
</Node>
</Plugin>
Make adjustments for your network as needed.
Run collectd using:
# service collectd start
Some useful information that is required while setting up:
Default directories for collectd:
Plugins -/opt/collectd/lib/collectd
Binaries - /opt/collectd/sbin/collectd
Configuration file: /opt/collectd/etc/collectd.conf
Run collectd like this:
/opt/collectd/sbin/collectd -C /opt/collectd/etc/collectd.conf
Test Collectd configuration:
#collectd -t
Test Collectd plugin configuration:
#collectd -T
Check netstat output:
# nestat -naptul |grep "25826"
Setting up Collectd agent
Now, we are going install the collectd agent on the client machine and then tell it to send the metrics to the collectd server (not Graphite). The collectd clients do not need "write_graphite" plugin and can use the older version of Collectd rpms that are available in CentOS/SL repositories. So, on each client, run:
# yum install collectd collectd-utils
Modify /etc/collectd.conf config file as per your requirement:
Hostname "hostname"
FQDNLookup true
BaseDir "/var/lib/collectd"
PIDFile "/var/run/collectd.pid"
PluginDir "/usr/lib/collectd"
TypesDB "/usr/share/collectd/types.db"
Interval 10
#Timeout 5
ReadThreads 5
LoadPlugin logfile
<Plugin logfile>
LogLevel info
File "/var/log/collectd.log"
Timestamp true
PrintSeverity true
</Plugin>
LoadPlugin network
<Plugin network>
Server "collectd-server.domain.com" "25826"
</Plugin>
LoadPlugin cpu
LoadPlugin load
LoadPlugin disk
LoadPlugin memory
LoadPlugin processes
Include "/etc/collectd/filters.conf"
Include "/etc/collectd/thresholds.conf"
Be sure to configure the network plugin with your collectd server information.
With this configuration, client metrics statistics are sent to collectd server on port 25826. These are further sent to Graphite. If you want to spice up web front-end, you can use grafana and show the trend of system metrics.
Enabling python plugin
If you wish to enable python and iptables plugin support, please do the following:
# yum install python-devel
# yum install iptables-devel
Now, re-compile the collectd source package once again for these modules.
# cd collectd-version
# ./configure --enable-python --enable-iptables
# make
# make install
This process is same for any additional plugins that you may wish to add - e.g. mysql, postgres etc.
Please check the Modules output carefully for the plugin support while configuring source package during compilation.
Some of the useful links that I encountered while setup:
- https://collectd.org/wiki/index.php
- https://collectd.org/wiki/index.php/Match:Hashed/Config
- http://blog.matthewdfuller.com/2014/06/sending-collectd-metrics-to-graphite.html
- https://keptenkurk.wordpress.com/2015/08/28/using-collectd-with-multiple-networked-instances/
- http://giovannitorres.me/enabling-almost-all-collectd-plugins-on-centos-6.html
