GlusterFS is a distributed file system which can be used to build volumes that span several hosts. It's used in a variety large scale cloud and web hosting applications.
A GlusterFS volume is a virtual disk which can be read and written from across a network. GlusterFS can be used to build high perfomance storage clusters that hold large volumes of data.
The data in GlusterFS volumes is divided into bricks, where each brick is a portion of a pysical drive that's used to store volume data. In practice a brick is just a directory whose contents are shared over the network.
Bricks can be replicated for improved fault tolerance. If one of the disks in a GlusterFS node dies, all the data in that drive's brick will be lost. If the cluster has been configured for replication, then any brick that is lost when a disk fails is available on another node. The broken disk just needs to be replaced and GlusterFS will automatically rebuild the lost brick from the contents of the other bricks.
Bricks can be distributed across more than one server to improve performance. This means that individual files may be written to different servers, even if those files are in the same directory. Distributing files helps to balance the load on the servers. If a volume contains a lot of files, spreading them across more than one server helps to spread the load when they are accessed.
GlusterFS supports striping where parts of a file can be written to different disks. Usually this is only recommended for very large files.
I installed Raspbian on three Banana Pis - one client, and two nodes for a simple storage cluster.
The client Banana Pi gets its IP address from the local DHCP server, and the Gluster nodes have static IP addresses - 192.168.0.30 and 192.168.0.31.
Connect to each node using ssh in different terminal windows:
And in another terminal:
I ran the bpi-config program on both Gluster nodes to expand the SD card's partition and change some basic settings like host names and passwords:
Next, I opened /etc/resolv.conf in nano, and entered DNS server IP addresses:
You can use Google's DNS servers for convenience:
Update Linux and install glusterfs-server:
Open /etc/hosts in a text editor on each node so that you can add the host names of the nodes:
In the hosts file on the first node, add these entries:
Add these entries to the host file on the second node:
You also need to open the hosts file on the client, and add both names:
On both nodes, you need to create directories to store the volume's contents:
In the ssh session for the first node, run this command to tell Gluster to form a server pool with the other node:
On the second node, use the gluster peer probe command to probe the first node:
You can run the next commands on either node. First you need to create the volume:
This tells gluster to create a volume called glusv0 with two bricks, where the contents of the brick on glus1 are replicated on glus2. Start the volume
You can get information about the volume with this command:
In order to check that everything is working, mount the volume from the command line of the first node:
Now create a file in the volume:
If you also mount the volume on the second node, the file should be accessible:
I set up another Banana Pi running Raspbian. Again I used bpi-config to set the hostname and password.
Update Linux and install glusterfs-client
You may need to restart the client Pi. To mount the volume from the client, run this command:
The -o option is used to pass the backupvolfile parameter which sets an alternate host name to access the volume in case the first server goes down. The client will automatically re-mount the volume via glus2.
In theory adding the following line to the client's fstab file should make the client mount the GlusterFS share at boot:
This didn't work because the GlusterFS client wasn't running when the fstab file was processed. Instead, I opened root's crontab file so that I could add a command to mount the share at reboot. This command opens the crontab file:
Add this line, and press control-o and return to save changes, and control-x to quit from nano:
This will execute two commands when the server boots up: the first is just a 10 second delay to allow the GlusterFS daemon to boot, and the second command mounts the volume.
You may need to make your Pi wait longer before running mount. If your Pi doesn't mount the volume when it boots, try using 'sleep 15' instead. This isn't an ideal way to fix this problem, but it's ok for most uses.
Change the owner of the volume so that it can be accessed by user bananapi:
Now your Banana Pi powered storage cluster should be up and running.
Share this page: