chris moos's blog » couchdb

CouchDB and Pylons: User Registration and Login

Chris Moos — Sun, 22 Feb 2009 01:47:08 +0000

In the previous tutorial, we learned how to get CouchDB and Pylons up and running, as well as create a simple page counter. Now we are going to implement a simple user authentication system. This tutorial will teach you how to use formencode to validate forms and CouchDB to store our user data.

Let’s start by creating a new pylons project and some controllers.

$ paster create -t pylons userdemo
$ cd userdemo
$ paster controller main
$ paster controller auth

Also, delete public/index.html.

For our controller main, we are going to add one action called index. This will be the main page for the site and will only be accessible for logged in users. If a user is not logged in, they will be redirected to the login page. Let’s add some routes for the main page, login, logout, and registration. Open up config/routing.py.

    map.connect('/', controller='main', action='index')
    map.connect('/auth/login', controller='auth', action='login', conditions=dict(method=['GET']))
    map.connect('/auth/login', controller='auth', action='login_post', conditions=dict(method=['POST']))
    map.connect('/auth/logout', controller='auth', action='logout')
    map.connect('/auth/register', controller='auth', action='register', conditions=dict(method=['GET']))
    map.connect('/auth/register', controller='auth', action='register_post', conditions=dict(method=['POST']))

For checking to see if a user is logged in, we will store a user_id in the session. Let’s make a decorator that we can add to our index action that will check to see if a user is logged in. If the user isn’t logged in, it will redirect to the login page. Your controllers/main.py should look have this:

from decorator import decorator
 
def require_login(func, *args, **kwargs):
    """ Checks to see if user_id is in session """
    if not 'user_id' in session:
        redirect_to('/auth/login')
    return func(*args, **kwargs)
require_login = decorator(require_login)
 
class MainController(BaseController):
 
    @require_login
    def index(self):
        return 'You are logged in! Click here to logout.'

If you start the server now and go to http://localhost:5000 you will be redirected to /auth/login. Good. Let’s get into CouchDB now…

Open up http://localhost:5984/_utils and create a new database, and call it userdemo.

Now we are going to define our User schema in model/__init__.py. A schema describes a certain type of document, and in this case, it will be a User document. This example is very simple, so we only need username, password, and salt. Note the other field, type. This will specify that the document is for users, and it will be automatically set with the default parameter when we store a document. We are also going to create a simple helper function that will return an instance of our database(from couchdb-python).

from couchdb import Server
from couchdb import schema
 
def get_db():
    server = Server('http://localhost:5984/')
    return server['userdemo']
 
class User(schema.Document):
    """ Simple user document """
    username = schema.TextField()
    password = schema.TextField()
    salt = schema.TextField()
    type = schema.TextField(default='user')

Next, we are going to create a simple template for auth/login. This will prompt the user to login, or register if the user does not have an account. (I assume you are using mako for your template engine).

templates/login.mak:

>
>>Login>>
>
>Login>
 action="/auth/login" method="post">
>
>>Username:>>${h.text('username')}>>
>>Password:>>${h.password('password')}>>
>
 type="submit" value="Login" />
>
 
% if c.invalid_user:
>*** An invalid username or password was entered.>
% endif
 
>Don't have an account? Click  href="/auth/register">here> to register.>
>>

Notice that we used h.text() and h.password(). These helpers create our input boxes and will also display an error when we get into form validation. Make sure to import those functions in lib/helpers.py.

from webhelpers.html.tags import text, password

Now that we’ve created or login template, let’s implement our action auth/login in controllers/auth.py.

class AuthController(BaseController):
 
    def login(self):
        return render('login.mak')

Okay, let’s run the server and try to go to http://localhost:5000. We are redirect to our new login page.

$ paster serve --reload development.ini

Okay, now let’s implement our login action. This is under controllers/auth.py, and the action is login_post. The first thing we will do is add a formencode schema for our login form. This will make sure the username and/or password is not empty, and it will display an error accordingly. Here is what our new auth.py file might look like.

import formencode
 
class LoginForm(formencode.Schema):
    username = formencode.validators.String(not_empty=True)
    password = formencode.validators.String(not_empty=True)
 
class AuthController(BaseController):
 
    def login(self):
        return render('login.mak')
 
class AuthController(BaseController):
 
    def login(self):
        return render('login.mak')
 
    def login_post(self):
        try:
            form_result = LoginForm().to_python(request.POST)
            try:
                user = authenticate_user(form_result['username'], form_result['password'])
                session['user_id'] = user.id
                session.save()
                redirect_to('/')
            except InvalidUser:
                c.invalid_user = True
                return render('login.mak')
        except formencode.Invalid, err:
            html = render('login.mak')
            return formencode.htmlfill.render(html, errors=err.error_dict)

This login_post action first checks to see if the form is valid, if not, it will display the login form with the appropriate errors. If the form is valid, we attempt to call the function authenticate_user, which will return a valid User document if the login is successful, or raise an Exception if the login information is invalid. If the login is valid, we save the user.id in our session. If not, we set the invalid_user context variable and return the login page.

Okay, so we have our login_post action defined, but we don’t have an authenticate_user method yet. Let’s implement that. But before we do that, we need to know a little bit more about CouchDB views. A view is a way to query data from our database. A view is defined by implementing map and reduce functions.

Let’s start by adding a user view, that will let us query user data. To create the view, go to http://localhost:5984/_utils and select the userdemo database. Go to Create Document. We are going to create a design document which is where the views are defined. The document ID should be _design/user. This view is accessible via http://localhost:5984/userdemo/_view/user/VIEW_FUNC. To define our view functions, open the newly created _design/user document and add a field called views. For now, just put {} for the value. Hit save document.

Each view function defines a map and optionally a reduce function. This lets us limit and control what documents our view will return. For now, we just want a simple view that allows us to query the database and get a user by the username. Remember that CouchDB returns data with a key/value. The key we want to return is the username.

Let’s talk about map for a second. A map function is passed a CouchDB document, and then emits, or adds, key/value pairs. CouchDB uses javascript as the default view server. We will call our view function by_username, and it will return the username as the key and the user document as the value. Open your _design/user document and put this in for the views field.

{
    "by_username": {
        "map": "function(doc) { if(doc.type == 'user') emit(doc.username, doc); }"
    }
}

CouchDB should look like this:

Notice that we check doc.type. This is because all documents are stored under one namespace, so we need a way to differentiate between different documents. We do this by setting a type field for every user, with the value “user”. You can access this view by going here: http://localhost:5984/userdemo/_view/user/by_username.

Alright, now that we have our view defined, let’s implement the authenticate_user function. We also create a custom exception class that we use for an invalid user. We use hashlib to generate a sha256 hash of the user’s password and a random salt. gen_hash_password is used later for our registration functions, for creating a new salt and getting a hash. Add this above your controller in controllers/auth.py.

from userdemo.model import get_db, User
import hashlib
 
class InvalidUser(Exception):
    pass
 
def hash_password(password, salt):
    m = hashlib.sha256()
    m.update(password)
    m.update(salt)
    return m.hexdigest()
 
def gen_hash_password(password):
    import random
    letters = 'abcdefghijklmnopqrstuvwxyz0123456789'
    p = ''
    random.seed()
    for x in range(32):
        p += letters[random.randint(0, len(letters)-1)]
    return hash_password(password, p), p
 
def authenticate_user(username, password):
    result = User.view(get_db(), '_view/user/by_username', key=username)
    if len(result) == 0:
        raise InvalidUser('bad username')
 
    user = result.__iter__().next()
 
    # check password
    if not hash_password(password, user.salt) == user.password:
        raise InvalidUser('bad password')
 
    return user

Okay, now our site can login a user. Let’s get into registration. Create the following functions for our registration action in controllers/auth.py.

def register(self):
        return render('register.mak')
 
    def register_post(self):
        try:
            form_result = RegisterForm().to_python(request.POST)
            user = User()
            user.username = form_result['username']
            pwd, salt = gen_hash_password(form_result['password'])
            user.password = pwd
            user.salt = salt
            user.store(get_db())
 
            return 'You are registered. Click here to login.'
        except formencode.Invalid, err:
            html = render('register.mak')
            return formencode.htmlfill.render(html, errors=err.error_dict)

These functions are similar to the login ones, except here we store a new user into the database, using the User schema document class that we defined in model/__init__.py. We also need to implement the RegisterForm, and also a custom validator, which checks to see if a username is taken.

def user_exists(username):
    result = User.view(get_db(), '_view/user/by_username', key=username)
    if len(result) == 0:
        return False
    return True
 
class UsernameValidator(formencode.validators.String):
    def validate_python(self, value, state):
        if user_exists(value):
            raise formencode.Invalid('Username already taken', value, state)
 
class RegisterForm(formencode.Schema):
    username = UsernameValidator(not_empty=True)
    password = formencode.validators.String(not_empty=True)

And that’s it! You can now register an account, login, and view the protected page(with the require_login decorator). Now you just need to add require_login to any function that is only for logged in users. In the next tutorial, I will go into some more advanced CouchDB topics and some cool map/reduce views.

You can download the pylons userdemo project here.

CouchDB and Pylons: Getting Started

Chris Moos — Wed, 04 Feb 2009 01:25:33 +0000

CouchDB is very cool(it’s built on erlang), and with Pylons, it is even cooler.

Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API.

Normally a Pylons web application will use some sort of RDBMS for storing data and persistence — such as MySQL or PostgreSQL. I’ve decided to go a different route and integrate Pylons with CouchDB. Right now there is a Python library, couchdb-python, that will help us communicate with our CouchDB’s HTTP/JSON api.

For this article, I will assume you have Pylons installed and you are somewhat familiar with it. We are going to create a simple page counter application.

CouchDB is written in Erlang, a functional programming language with high concurrency. Let’s install Erlang.

$ wget http://erlang.org/download/otp_src_R12B-5.tar.gz
$ tar -xvzf otp_src_R12B-5.tar.gz
$ cd otp_src_R12B-5
$ ./configure
$ sudo make && make install

You should now have erlang installed. If you type erl and hit enter, you should see something like this.

$ erl
Erlang (BEAM) emulator version 5.6.3 [source] [smp:2] [async-threads:0] [kernel-poll:false]
 
Eshell V5.6.3  (abort with ^G)
1>

Now let’s install CouchDB. The latest version can be found on this page.

$ wget http://download.filehat.com/apache/incubator/couchdb/0.8.1-incubating/apache-couchdb-0.8.1-incubating.tar.gz
$ tar -xvzf apache-couchdb-0.8.1-incubating.tar.gz
$ cd apache-couchdb-0.8.1-incubating
$ ./configure
$ sudo make && make install

Okay. CouchDB should now be installed. Let’s start it up.

$ couchdb
Apache CouchDB 0.9.0a721128-incubating (LogLevel=info) is starting.
Apache CouchDB has started. Time to relax.

Alright, we are almost there. Let’s install the couchdb-python library.

easy_install:

$ sudo easy_install CouchDB==0.5

or the latest development version:

$ svn checkout http://couchdb-python.googlecode.com/svn/trunk/ couchdb-python-read-only
$ cd couchdb-python-read-only
$ sudo python setup.py install

Now that we’ve got CouchDB all setup and running, let’s open up the web interface and create a database. CouchDB has a great web interface, located at http://localhost:5984/_utils/.

When you are at the web interface, go to create database, and let’s name it tutorial. After you have created the new database, you are redirected and are shown an empty database with no documents.

We now have created our CouchDB database and we are ready to start adding new documents.

Let’s create a new Pylons project named tut1, delete the default index.html, and a create a controller called main.

$ paster create -t pylons tut1
$ cd tut1
$ rm tut1/public/index.html
$ paster controller main

For this tutorial, we aren’t really going to go into too much detail. We are just going to make one page with a simple page counter. The next tutorial will be about some more advanced topics.

Okay, open up your config/routing.py file so we can add a route to our index page in the main controller.

map.connect('/', controller='main', action='index')
map.connect('/{controller}/{action}')
map.connect('/{controller}/{action}/{id}')

Now let’s open up the file controllers/main.py. Let’s add an import for the couchdb-python module.

import couchdb.client

For the counter to work, we are going to just create a simple document that will store the visits.

Here is what our method for index will look like.

    def index(self):
        db = couchdb.client.Database('http://localhost:5984/tutorial')
 
        # if counter does not exist, let's initialize it
        if not 'counter' in db:
            db['counter'] = {'visits': 1}
            return 'You are the first user!'
        else:
            counter = db['counter']
            counter['visits'] = counter['visits'] + 1
 
            # save
            db['counter'] = counter
 
            return 'You are user #%d.' % (counter['visits'])

First, a server object is created, and then the database is selected. After, we check to see if the counter document exists, and if it doesn’t, we initialize it. CouchDB stores a documents as a dictionary. A document can have as many key/value pairs as it wants. For this example, we just have one key, visits, which stores an integer.

Now that our index method is done, let’s run our pylons application.

$ paster serve --reload development.ini

Open up the site in a web browser, at http://localhost:5000.

You should see the following on your first visit.

You are the first user!

And on subsequent visits…

You are user #2.

You are user #3.

You have successfully created a CouchDB powered page counter! Now, this isn’t meant to be a production page counter, but it should give you a basic example of how CouchDB stores and retrieves documents.

In the next tutorial, we will integrate CouchDB with Pylons more, use Forms, CouchDB-Python’s Schemas, and more.

MoosTrax will return.

Chris Moos — Sun, 25 Jan 2009 05:21:50 +0000

Yes, I recently made a post saying that MoosTrax will be going away forever. Due to all the comments and feedback, it is obvious to me that there are many people that enjoyed the service — so I’m going to bring it back.

I’m moving my database backend from CouchDB to MySQL, so my issues with CouchDB won’t prevent me from providing MoosTrax. I have been working a lot on migrating the site and fixing it up…so I really hope to have it up in the next week or so.

In the mean time, I’m also developing an Android client, so that those of you with a T-Mobile G1(myself included) will be able to use MoosTrax. Screenshot below.

After the site is up and running and the Android client is stable, I want to make an iPhone client the next priority. Also, if there are enough people that want a Windows Mobile client, I would consider that as well. Please e-mail moostraxsupport@gmail.com if you are interested in a Windows Mobile client.

Hasta la vista, MoosTrax.

Chris Moos — Tue, 20 Jan 2009 23:42:56 +0000

UPDATE 01/21/09:

Okay, okay. A lot of you seem to like the site…I’m going to bring it back. Moving back to MySQL though. Stay tuned.

Well, it was a fun while it lasted. Unfortunately, I’ve decided to close the doors of MoosTrax permanently.

It started with BlackBerryTracker.com, which I had set up in Fall of 2007, to provide a site to track BlackBerry GPS-enabled devices. Prior to starting the site I had got my first BlackBerry, the 8800, and also my first GPS enabled phone. I decided to to learn a little J2ME and BlackBerry device programming, while trying out the GPS functionality of the phone. The end product of this was BlackBerryTracker which was relatively well received by the community and I believe provided a good service to a lot of people. The site grew to just over 10,000 people in the following year.

After this, I decided to redesign the site, use a new database system(CouchDB), and choose a new, less specific name(MoosTrax). I was very happy with the new site, and especially the database, CouchDB, because of its less relational nature and document-oriented architecture(and who doesn’t love map/reduce).

Unfortunately, with CouchDB came extremely large database sizes. So large, in fact, that my 80GB hard drive became quickly filled. It filled so fast that I couldn’t even compact it, or make it smaller. My only option at this point was to move the 80GB database to another, remote server, compact it, delete it from the server, and replace it with the smaller copy. This proved to be more work than it was worth, and my interests in MoosTrax had been declining, so I have now decided to end it all here.

I thank everyone who helped me in testing BlackBerryTracker/MoosTrax and those who supported the progress of the site!

It was a great run and I hope to begin working a new project that will hopefully serve a new community.