Squid proxy for CouchDB
I’m playing a bit with CouchDB, document-oriented, no-sql database with HTTP REST api. I had to test how to scale it and load-balance high web traffic to n CouchDB nodes.
CouchDB is very cool itself, but when you ask for documentation.. Let’s say that sometimes it’s hard to find right page on wiki. For example, replication page lacks information about really cool feature – continuous replication! So don’t forget to look at other sources, like the book CouchDB: The Definitive Guide, in this case replication chapter.
Based on book content I’ve designed simple multi-node architecture with one master node and many slaves:
In fact I had 4 machines: Squid proxy (feed, 192.168.100.1), CouchDB master node (cdb0, 192.168.100.2) and 2 CouchDB slave nodes (cdb1 192.168.100.3, cdb2 192.168.100.4). I had to configure feed to route all PUT, POST and DELETE requests to cdb0 node, and load-balance all GET request between cdb1 and cdb2. I’ve used Ubuntu 9.10 RC1 since it has CouchDB 0.10 by default.
Let’s start by installing Squid on feed host:
sudo apt-get install squid3Next change it’s configuration:
sudo gedit /etc/squid3/squid.confChange line line http_port to:
http_port 3128 defaultsite=feed vhostNext, in cache_peer section, write
cache_peer 192.168.100.2 parent 5984 0 no-query originserver name=master acl master_acl method POST PUT DELETE cache_peer_access master allow master_acl cache_peer 192.168.100.3 parent 5984 0 no-query originserver round-robin name=slave1 cache_peer 192.168.100.4 parent 5984 0 no-query originserver round-robin name=slave2 acl slave_acl method GET cache_peer_access slave1 allow slave_acl cache_peer_access slave2 allow slave_aclHuh, looks complicated? In fact it is only definition of two access policies (master and slave) and load balancing it.
Next step would be installing 3xUbuntu 9.10 with CouchDB enabled at port 5984. I assume that it’s the easiest part 
.
To enable replication, install on CouchDB slave nodes execute:
curl -X POST http://localhost:5984/_replicate -d ‘{“source”:”http://192.168.100.2:5984/replication_test”, “target”:”replication_test”, “continuous”:true}’
To test it simply open your web browser and poitn to http://localhost:3128/_utils to see if futon is working. Your requests should be load-balanced
Future improvements:
* Test “accel” option of http_port tag to enable cache
* Find out why tests in Futon are failing (I’m mainly interested in testing OAuth)
Comments
2 Responses to “Squid proxy for CouchDB”
Leave a Reply
Próbowałeś może MongoDB?
Nie, ale słyszałem i czytałem trochę. Ogólnie CouchDB ma fajne koncepty – jak Map Reduce, zewnetrzne handlery przez co łatwo się robi dodatki (w stylu wyszukiwanie pełnotekstowe przez apache lucene) czy klastrowanie. Ogólnie to nie mój wybór, ale jest ciekawy