CouchDB and Pylons: Getting Started
CouchDB is very cool(it’s built on erlang), and with Pylons, it is even cooler.
Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API.
Normally a Pylons web application will use some sort of RDBMS for storing data and persistence — such as MySQL or PostgreSQL. I’ve decided to go a different route and integrate Pylons with CouchDB. Right now there is a Python library, couchdb-python, that will help us communicate with our CouchDB’s HTTP/JSON api.
For this article, I will assume you have Pylons installed and you are somewhat familiar with it. We are going to create a simple page counter application.
CouchDB is written in Erlang, a functional programming language with high concurrency. Let’s install Erlang.
$ wget http://erlang.org/download/otp_src_R12B-5.tar.gz $ tar -xvzf otp_src_R12B-5.tar.gz $ cd otp_src_R12B-5 $ ./configure $ sudo make && make install
You should now have erlang installed. If you type erl and hit enter, you should see something like this.
$ erl Erlang (BEAM) emulator version 5.6.3 [source] [smp:2] [async-threads:0] [kernel-poll:false] Eshell V5.6.3 (abort with ^G) 1>
Now let’s install CouchDB. The latest version can be found on this page.
$ wget http://download.filehat.com/apache/incubator/couchdb/0.8.1-incubating/apache-couchdb-0.8.1-incubating.tar.gz $ tar -xvzf apache-couchdb-0.8.1-incubating.tar.gz $ cd apache-couchdb-0.8.1-incubating $ ./configure $ sudo make && make install
Okay. CouchDB should now be installed. Let’s start it up.
$ couchdb Apache CouchDB 0.9.0a721128-incubating (LogLevel=info) is starting. Apache CouchDB has started. Time to relax.
Alright, we are almost there. Let’s install the couchdb-python library.
easy_install:
$ sudo easy_install CouchDB==0.5
or the latest development version:
$ svn checkout http://couchdb-python.googlecode.com/svn/trunk/ couchdb-python-read-only $ cd couchdb-python-read-only $ sudo python setup.py install
Now that we’ve got CouchDB all setup and running, let’s open up the web interface and create a database. CouchDB has a great web interface, located at http://localhost:5984/_utils/.
When you are at the web interface, go to create database, and let’s name it tutorial. After you have created the new database, you are redirected and are shown an empty database with no documents.
We now have created our CouchDB database and we are ready to start adding new documents.
Let’s create a new Pylons project named tut1, delete the default index.html, and a create a controller called main.
$ paster create -t pylons tut1 $ cd tut1 $ rm tut1/public/index.html $ paster controller main
For this tutorial, we aren’t really going to go into too much detail. We are just going to make one page with a simple page counter. The next tutorial will be about some more advanced topics.
Okay, open up your config/routing.py file so we can add a route to our index page in the main controller.
map.connect('/', controller='main', action='index') map.connect('/{controller}/{action}') map.connect('/{controller}/{action}/{id}')
Now let’s open up the file controllers/main.py. Let’s add an import for the couchdb-python module.
import couchdb.client
For the counter to work, we are going to just create a simple document that will store the visits.
Here is what our method for index will look like.
def index(self): db = couchdb.client.Database('http://localhost:5984/tutorial') # if counter does not exist, let's initialize it if not 'counter' in db: db['counter'] = {'visits': 1} return 'You are the first user!' else: counter = db['counter'] counter['visits'] = counter['visits'] + 1 # save db['counter'] = counter return 'You are user #%d.' % (counter['visits'])
First, a server object is created, and then the database is selected. After, we check to see if the counter document exists, and if it doesn’t, we initialize it. CouchDB stores a documents as a dictionary. A document can have as many key/value pairs as it wants. For this example, we just have one key, visits, which stores an integer.
Now that our index method is done, let’s run our pylons application.
$ paster serve --reload development.iniOpen up the site in a web browser, at http://localhost:5000.
You should see the following on your first visit.
You are the first user!
And on subsequent visits…
You are user #2.
You are user #3.
You have successfully created a CouchDB powered page counter! Now, this isn’t meant to be a production page counter, but it should give you a basic example of how CouchDB stores and retrieves documents.
In the next tutorial, we will integrate CouchDB with Pylons more, use Forms, CouchDB-Python’s Schemas, and more.
[...] Normally a Pylons web application will use some sort of RDBMS for storing data and persistence — such as MySQL or PostgreSQL. I’ve decided to go a different route and integrate Pylons with CouchDB Read more [...]
Very good, how about a tutorial for a simple facebook app using python?
Thanks for the tutorial.
db = couchdb.client.Database(‘http://localhost:5984/tutorial’) did not create db.
To make the code work for me I added in my imports:
from couchdb import Server
immediately following db = couchdb.client.Database(‘http://localhost:5984/tutorial’)
I added:
if not db:
db = Server(‘http://localhost:5984/’).create(‘tutorial’)
Then everything went fine. From a brief bit of looking around it seems that couchdb.client.Database accesses a couchdb database but I don’t think that it creates one if it doesn’t exist. I’m using couchdb 0.8.1 and couchdb-python 0.5
These are two very cool bits of technology and your post is really well written. Can’t wait to read the next installment
[...] the previous tutorial, we learned how to get CouchDB and Pylons up and running, as well as create a simple page counter. [...]
[...] CoungDB with Pylons [...]
Great write up, thanks.
Any thoughts on how to deal with concurrent requests though? with SQL dbs, we can have transactions and row locking to ensure the counter is in a valid state. Can this be acheived with couchdb backend? and if so, how scalable would it be?
upattwothirty:
I can think of two ways you could approach this.
1. Instead of just incrementing a single value for the counter example, you could create a new document for each visit, then run a map/reduce view to return the total visits.
2. When you update a document in couchdb, it will also send the revision of the document. If you try to update the document, and CouchDB detects a different revision(example: someone else changes the document), it will return an error. You would probably handle this in your application code somewhere, and retry the “transaction”.
As far as scalability….I’m not sure
. Wait for a 1.0 release when replication etc,. is all hammered out.
From the CouchDB docs:
To update an existing document, you also issue a PUT request. In this case, the JSON body must contain a _rev property, which lets CouchDB know which revision the edits are based on. If the revision of the document currently stored in the database doesn’t match, then a 412 conflict error is returned.
@upattwothirty: Short answer, don’t have two people update the same document. If you have something that needs adding to, like adding comments to a blog, add each comment as its own CouchDB document rather than extending an array inside a single Blog entry CouchDB doc.
Note that the entire PylonsHQ website is also running on CouchDB + Pylons, and is open-sourced: http://bitbucket.org/bbangert/kai/
It uses the raw couchdb access as shown above, in addition to using python-couchdb schema docs which allow it to coerce some values from JSON (like datetime) back to a proper Python form.