Mar 18, 2011

Configuring repcached service on Debian/Ubuntu

Repcached is a slightly modified version of memcached, that supports replication of data between two repcached nodes.

Let's say you've read all about these two and you know the benefits of replication and why this article could be useful to you.

One example of why replicated memcache could be useful are replicated PHP sessions between servers.

If you want to configure that PHP sessions are stored in memcache's memory, you need to edit these settings in /etc/php5/apache2/php.ini:
session.save_handler = memcache
session.save_path = "tcp://IP_OF_REPCACHE_1:11311, tcp://IP_OF_REPCACHE_2:11311"
and these optional settings in /etc/php5/apache2/conf.d /memcache.ini:
memcache.maxratio=0
memcache.allow_failover=1
memcache.allow_failover setting is used if one of the servers becomes unreachable, so there is an automatic failover.
Read more about configuring PHP sessions in memcached here.

Now, let's set up repcached to start on boot and System V init scripts, so you can easly start and stop the daemon with the service command.

Steps described here imitate memcached's default configuration in great detail, so you shoud set up memcached before repcached.
sudo apt-get install memcached

Obtain, configure, compile and install repcached. There is a dependancy with libevent-dev for repcached.
sudo apt-get install libevent-dev
tar xvf memcached-1.2.8-repcached-2.2.tar 
cd memcached-1.2.8-repcached-2.2/
./configure --enable-replication
make
make install

(Read this if it won't compile)

At this point you have two installations of memcached. Default memcached that came from apt packages, which is installed in /usr/bin/memcached and repcached, that installed itself in /usr/local/bin/memcached, leaving the original memcached intact.

Now that we have both versions installed, we can copy memcached's default settings and init script and modify them to use repcached. This way you can quickly switch between versions. I would even recommend using default ports (just remember to firewall them!) Arguments are saved in /etc/memcached.conf, so we will create /etc/repcached.conf

See example here.

Note that the only differences with memcached.conf is the name (repcached) and two extra arguments: -x for the server IP and -X for replication port.

Memcached has an enable/disable config in /etc/default so you can quickly switch between daemons or disable them. We will copy this as well.
cp /etc/default/memcached /etc/default/repcached
vi /etc/default/repcached
Change the line to: ENABLE_REPCACHED=yes, and then edit /etc/default/memcached
vi /etc/default/memcached
and disable it, by changing the line to ENABLE_MEMCACHED=no.

Now let's move on to init scripts.
cd /etc/init.d
cp memcached repcached
Edit the file /etc/init.d/repcached.

Here is my example.

Again, we didn't change much, mostly changed from memcached to repcached, but note that the actual start-up of the service happens in this file: /usr/share/memcached/scripts/start-repcached which doesn't exist yet, so we will copy and edit it.
cp /usr/share/memcached/scripts/start-memcached /usr/share/memcached/scripts/start-repcached
File contents or /usr/share/memcached/scripts/start-repcached

Setting up repcached to start at boot

We need to be sure that /etc/init.d/repcached is executable. If you copied it from memcached, everything should be OK, but if init's not recognising the repcached service, you need to chmod +x /etc/init.d/repcached

After you've run update-rc.d command in the terminal it will create shortcuts in rc?.d files which are read at boot.
update-rc.d repcached defaults
For more information on update-rc.d, click here.

You have successfully configured repcached as a service and to start on boot.

To start/stop repcached use
service repcached start
service repcached stop
Try to run repcached by hand at first with the configuration you provided in /etc/repcached.conf.
In my example it's this:
/usr/local/bin/memcached -m 64 -p 11211 -u memcache -X 11212 -x 22.163.130.33

After installing repcached on another machine I've found out that the default user for memcached is nobody, not memcache, so please always check the differences from the default memcache config with the repcached config you've modified or copied from here.

4 comments:

Rajeev Nair said...

Hi Jean,
Thank you very much for the tutorial. This is awesome and I got my repcache working within an hour (amazing). I have been trying to do this with other tutorials forever and it would never work. I have one question though. -x option in repcached.conf - what if I have more than two servers in my repcached cluster and I would like to mention more than one peer server

jeancaffou said...

Repcached supports only two servers, configuring a cyclic replication (like in MySQL) won't work, because repcached doesn't replicate "rep" command.

Greg said...

Hello,

I configured 3 servers: 1 Apache and 2 Repcached. If 1 repcached goes off, will it recover sessions created on the other repcached when it gets back online?
It looks like it doesn't but then if I stop the second repcached (e.g. for maintenance), I will loose sessions, no?

Thanks,

Greg.

jeancaffou said...

Yes, that's the caveat. When s1 comes back online it will not recover data. Replication happens on "set". If s2 drops you loose sessions completely.