Skip to navigation

Pyplate Multi-Site

I plan to scale the Banana Pi cluster in order to cope with more traffic.  I also want to use it to host more than one web site.  Before I scale up, I need to prepare the Pyplate CMS to run on a larger architecture.  I want to set up a mass blogging platform where any server in the cluster to be able to serve any page from any site, a bit like Tumblr or Blogger.

Pyplate was originally developed to serve one site per installation.  I've modified it so that a single installation can now serve serveral sites from the same server.  Each site has its own templates, its own web root directory and database tables, but all sites share the same Python application code in /usr/share/pyplate/wsgi-scripts.  Once several sites have been set up on one server, replicating those sites across a cluster is relatively simple.

When a server receives a request, it now has to determine which site the requested page belongs to.  It does this by checking each HTTP request's host field.  This field contains the domain name of the site, which is used to work out the path to the site data.

All of a Pyplate site's information used to be stored in /usr/share/pyplate like this:

/usr/share/pyplate/
 |_ backup
 |_ content
 |_ database
 |_ sample_configs
 |_ template
 |_ themes
 |_ wsgi-scripts

Now /usr/share/pyplate contains a directory for each site in the cluster:

/usr/share/pyplate
 |_ 192.168.0.6
 |_ banoffeepiserver.com
 |_ blog.pyplate.com
 |_ linuxwebservers.net
 |_ wsgi-scripts
 |_ www.pyplate.com

These site directories contain all the data for each Pyplate site:

/usr/share/pyplate/banoffeepiserver.com/
 |_ backup
 |_ content
 |_ database
 |_ sample_configs
 |_ template
 |_ themes
 |_ wsgi-scripts

Each site needs its own web root directory.  I usually use /var/www as the web root directory, but now I need to create a new directory for each site in /var/www.  I modified Nginx's default server block in /etc/nginx/sites-available/default (equivalent to a virtual host in Apache) so that the $host variable is used in the root directive:

root /var/www/$host;

Now Nginx will append the host field in HTTP requests to /var/www to generate the docment root.  This approach means I can use one generic configuration file to serve each site. This is a good thing because it means I don't have to create a new configuration file for each new site that I create.

Now the /var/www directory contains the web root for each of the sites on that server:

/var/www
 |_ 192.168.0.6
 |_ banoffeepiserver.com
 |_ blog.pyplate.com
 |_ linuxwebservers.net
 |_ www.pyplate.com

Pyplate needs to be able to access each sites' files.  Pyplate.py contains functions named getCMSRoot and getWebRoot to return the path to a site's files and web root directory. These functions have been modified so that they now append the name of the site to the path that they return:

# get CMS path
# return CSM root directory
#
def getCMSRoot ():
    global host_name
    return "/usr/share/pyplate/%s" % host_name

# getWebRoot
# return web server root directory
#
def getWebRoot ():
    return "/var/www/%s" % host_name

Until now, I've used SQLite to store data in Pyplate sites. I want to use a network database that supports replication, so I modified Pyplate to work with MySQL.  This means I can now put the database and web servers on seperate clusters for increased efficiency.  

There is still one database, but it now contains data for several sites.  There is a set of tables for each site, and each set is prefiexed with the name of the site.  In SQL, '.' is a special character, so any dot in the site name needs to be converted into an '_'.  

When an instance of the database object is created, a site prefix is passed to the class constructor:

    def __init__(self, hostname, username, password, dbname, prefix):
        self.conn = MySQLdb.connect(hostname, username, password, dbname)
        self.curs = self.conn.cursor()
        self.prefix = prefix.replace('.', '_')

In subsequent calls using the database object, queries are formatted with the prefix at the start of the table name:

    def get_page_from_db(self, uri_path):

        query = "SELECT * FROM {prefix}_pages WHERE path= (%s)"
        sql_str = self.prefix_format(query)

        self.curs.execute(sql_str, (uri_path,))

        row=self.curs.fetchone()

        return row

Importing data

Pyplate databases can be exported to a set of XML files.  I took the XML files from the SQLite version of banoffeepiserver.com, and imported the data into a MySQL database.  The tables where created with the prefix banoffeepiserver_com.  

I imported the data using a command line utility that I wrote.  The script can also be used to drop tables and set a random admin password.  The usage information for this tool is as follows:

Usage:
create_pyplate_db.py -site <site name> [options]
 -c    --create      create pyplate tables
 -d    --drop        drop pyplate tables
 -i    --init        init directories for a new site
 -p    --password    generate a password amd save it in the database
 -s    --site        site name used as prefix for database tables

This is the command that I used to import the data:

./create_pyplate_db.py -s banoffeepiserver.com -c -p
Setting up database...banoffeepiserver.com
/usr/share/pyplate/banoffeepiserver.com
banoffeepiserver.com
Connected to database
Creating tables
Created tables
Committed tables
importing data
done importing data

This updated version of Pyplate is not available yet.  It still needs to be tested more thoroughly, and there are a few UI issues that need to be addressed.  In time it will be available to download from www.pyplate.com.

Share this page:

comments powered by Disqus