Skip to navigation

Installing Ganglia on a Banana Pi

I've been looking for tools to monitor my ARM server farm, and the best contender so far is Ganglia, which is a tool for monitoring large clusters of computers, and logging their performance over time.  You can see an online demo of the Ganglia web UI used to monitor Wikipedia's servers.

Ganglia has three main components:

  • Ganglia Meta Daemon, gmetad
  • Ganglia Monitor Daemon, gmond
  • the Ganaglia web front end

The monitor needs to be installed and configured on each node that's being monitored. The meta daemon needs to be installed on a master node, where it will collect data from the monitors on the other nodes.  Data is transferred using UDP multicasting.  The multicast addresses used can be specified in the Ganglion configuration files. 

Installing Ganglia

To begin with, I'm just going to install Ganglia on a single Banana Pi.  I'm going to use the Ganglia web front end, so I'm going to install Apache and support for PHP5:

sudo apt-get install apache2 php5 libapache2-mod-php5 php5-json 

I'm not completely sure if php5-json is needed, but it's installed on my server.  Now install the necessary Ganglia components:

sudo apt-get install ganglia-frontendweb gmetad gmetad rrdtool ganglia-monitor

The configuration file for the Ganglia daemon is in /etc/ganglia/gmetad.conf, and the configuration file for the monitor is /etc/ganglia/gmond.conf. I used Leafpad to edit the monitor configuration file:

sudo leafpad /etc/ganglia/gmond.conf

and I set the name of the cluster:

...
cluster {
  cluster = "my cluster"
  owner = "unspecified"
  latlong = "unspecified"
  url = "unspecified"
}
...

Copy the Ganglia sample Apache configuration file to Apache's 'sites-available' directory, and enable the new configuration:

sudo cp /etc/ganglia-webfrontend/apache.conf /etc/apache2/sites-available/ganglia.conf
sudo a2ensite ganglia.conf

Restart Ganaglia monitor, Ganglia Meta Daemon and Apache:

sudo service ganglia-monitor restart ; sudo service gmetad restart ; sudo service apache2 restart

Now visit http://<your Banana Pi's IP address>/ganglia/ in a web browser, and you should see something like this:

One of the benefits of Ganglia is it means I don't have to have a web server installed on each node that I want to monitor, just the control node.

The downside of Ganglion is that it accumulates a signicant amount of data over time. The RRD tools database that's used to store the data has an averaging function which reduces the amount of space needed to store older data (but also reduces the resolution of that data), but the database still occupies quite a lot of space.  My Ganglia master node will need its own hard disk.

 

See also: http://ganglia.info/

Set up Ganglia on multiple clusters

Ganglia can be used to monitor groups of clusters.  Ganglia-monitor needs to be installed and configured on each node that's being monitored.  The Ganglia meta daemon (gmetad) runs on a master node and connects to the monitor processes to collect system information.  A group of clusters monitored by a single Ganglia server is called a grid.  

I recently built a database cluster and a file storage cluster, so I installed Ganglia-monitor on each node in these clusters.  I used two more Banana Pis as control nodes in a cluster of their own.

Set up the Ganglia server

I installed the following packages on the Ganglia master node:

sudo apt-get install apache2 php5 libapache2-mod-php5 php5-json 
sudo apt-get install ganglia-frontendweb gmetad gmetad rrdtool ganglia-monitor

The next step is to edit /etc/ganglia/gmetad.conf:

sudo nano /etc/ganglia/gmetad.conf

Ganglia uses different ports to distinguish between different clusters.  I'm using port 8650 for the database cluster, port 8655 for a small cluster of control nodes, and port 8656 for the cluster of cluster servers.  I've set up three data_source lines which specify the cluster names, refresh interval, and a list of hosts in each cluster:

data_source "cmd_cluster" 60 192.168.0.8:8655 192.168.0.9:8655
data_source "db_cluster" 60 192.168.0.35:8650 192.168.0.36:8650 192.168.0.37:8650 192.168.0.38:8650
data_source "g_cluster" 60 192.168.0.30:8656 192.168.0.31:8656 192.168.0.32:8656 192.168.0.33:8656

I uncommented the gridname directive and set it to "ARM_Farm":

gridname "ARM_Farm"

I also edited the list of trusted servers to include each IP address:

trusted_hosts 127.0.0.1 192.168.0.9  192.168.0.30 192.168.0.31 192.168.0.32 192.168.0.33 192.168.0.35 192.168.0.36 192.168.0.37 192.168.0.38

Next I copied the Ganglia Apache configuration file to Apache's config directories and enabled it:

sudo cp /etc/ganglia-webfrontend/apache.conf /etc/apache2/sites-available/ganglia.conf
sudo a2ensite ganglia.conf

Restart Apache, the monitor, and server processes:

sudo service ganglia-monitor restart ; sudo service gmetad restart ; sudo service apache2 restart

Set up the monitors

Install ganglia-monitor on each client:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install ganglia-monitor

Edit /etc/ganglia/gmond.conf:

sudo nano /etc/ganglia/gmond.conf

Enter the name of the cluster:

cluster { 
  name = "cmd_cluster" 
  owner = "unspecified" 
  latlong = "unspecified" 
  url = "unspecified" 
} 

Edit the port number for the udp send and recv channels, and for the tcp connection port:

/* Feel free to specify as many udp_send_channels as you like.  Gmond
   used to only support having a single channel */
udp_send_channel {
  mcast_join = 239.2.11.71
  port = 8650
  ttl = 1
}

/* You can specify as many udp_recv_channels as you like as well. */
udp_recv_channel {
  mcast_join = 239.2.11.71
  port = 8650
  bind = 239.2.11.71
}

/* You can specify as many tcp_accept_channels as you like to share
   an xml description of the state of the cluster */
tcp_accept_channel {
  port = 8650
}

Restart the ganglia-monitor process:

sudo service ganglia-monitor restart
Stopping Ganglia Monitor Daemon: gmond.
Starting Ganglia Monitor Daemon: gmond.

Unicast and multicast communication

There are some situations where UDP multicasting won't work.  The most common reason for this is that some ethernet switches don't support it.  Unicasting can be used in stead.  You need to comment out the lines with the mcast_join address, set the bind IP address to the server's IP address in the recv channel, and the host IP address (also the server's IP address) in the send channel.

/* Feel free to specify as many udp_send_channels as you like.  Gmond 

   used to only support having a single channel */ 
udp_send_channel { 
  host = 192.168.0.8
#  mcast_join = 239.2.11.71 
  port = 8655
  ttl = 1 
} 

/* You can specify as many udp_recv_channels as you like as well. */ 
udp_recv_channel { 

#  mcast_join = 239.2.11.71 
  port = 8655
  bind = 192.168.0.8
} 

/* You can specify as many tcp_accept_channels as you like to share 
   an xml description of the state of the cluster */ 
tcp_accept_channel { 
  port = 8655 
} 

You may have to wait a minute or two before all nodes show up in the UI, but eventually when you visit Ganglia's url in your browser (http://<your Banana Pi's IP address>/ganglia/), you should see the Ganglia grid overview (see screen shot above) in your browser.

Note that you can use the links in the top left hand of the page to move between different views and select individual nodes.

Detailed information is available about each node's resources.  These graphs show a node's CPU usage:

These graphs show information about a node's network utilization:

Share this page: