To do this, you start a thread and use python’s Queue for managing the tasks. Here is a very basic implementation:

config/environment.py:

# PYLONS IMPORTS
 
from myapp.lib.myworker import start_myworker
 
def load_environment(global_conf, app_conf):
    """Configure the Pylons environment via the ``pylons.config``
    object
    """
 
    # PYLONS STUFF GOES HERE
 
    # start worker
    start_myworker()

lib/myworker.py:

import Queue
import threading
 
worker_q = Queue.Queue()
 
class MyWorkerThread(threading.Thread):
    def run(self):
        print 'Worker thread is running.'
 
        while True:
            msg = worker_q.get()
            try:
                # do a long running task...
                print 'We got %s, do something with it!' % (msg)
            except Exception, e:
                print 'Unable to process in worker thread: ' + str(e)
            worker_q.task_done()                
 
def start_myworker():
    worker = MyWorkerThread()
    worker.start()

And in your controller….

from myapp.lib.myworker import worker_q
 
class WorkerController(BaseController):    
    def do_task(self):
      worker_q.put({'id': generate_some_unique_id(), 'msg': 'your data goes here'})

Let’s start by creating a new pylons project and some controllers.

$ paster create -t pylons userdemo
$ cd userdemo
$ paster controller main
$ paster controller auth

Also, delete public/index.html.

For our controller main, we are going to add one action called index. This will be the main page for the site and will only be accessible for logged in users. If a user is not logged in, they will be redirected to the login page. Let’s add some routes for the main page, login, logout, and registration. Open up config/routing.py.

    map.connect('/', controller='main', action='index')
    map.connect('/auth/login', controller='auth', action='login', conditions=dict(method=['GET']))
    map.connect('/auth/login', controller='auth', action='login_post', conditions=dict(method=['POST']))
    map.connect('/auth/logout', controller='auth', action='logout')
    map.connect('/auth/register', controller='auth', action='register', conditions=dict(method=['GET']))
    map.connect('/auth/register', controller='auth', action='register_post', conditions=dict(method=['POST']))

For checking to see if a user is logged in, we will store a user_id in the session. Let’s make a decorator that we can add to our index action that will check to see if a user is logged in. If the user isn’t logged in, it will redirect to the login page. Your controllers/main.py should look have this:

from decorator import decorator
 
def require_login(func, *args, **kwargs):
    """ Checks to see if user_id is in session """
    if not 'user_id' in session:
        redirect_to('/auth/login')
    return func(*args, **kwargs)
require_login = decorator(require_login)
 
class MainController(BaseController):
 
    @require_login
    def index(self):
        return 'You are logged in! Click <a href="/auth/logout">here</a> to logout.'

If you start the server now and go to http://localhost:5000 you will be redirected to /auth/login. Good. Let’s get into CouchDB now…

Open up http://localhost:5984/_utils and create a new database, and call it userdemo.

Now we are going to define our User schema in model/__init__.py. A schema describes a certain type of document, and in this case, it will be a User document. This example is very simple, so we only need username, password, and salt. Note the other field, type. This will specify that the document is for users, and it will be automatically set with the default parameter when we store a document. We are also going to create a simple helper function that will return an instance of our database(from couchdb-python).

from couchdb import Server
from couchdb import schema
 
def get_db():
    server = Server('http://localhost:5984/')
    return server['userdemo']
 
class User(schema.Document):
    """ Simple user document """
    username = schema.TextField()
    password = schema.TextField()
    salt = schema.TextField()
    type = schema.TextField(default='user')

Next, we are going to create a simple template for auth/login. This will prompt the user to login, or register if the user does not have an account. (I assume you are using mako for your template engine).

templates/login.mak:

<html>
<head><title>Login</title></head>
<body>
<h1>Login</h1>
<form action="/auth/login" method="post">
<table>
<tr><th>Username:</th><td>${h.text('username')}</td></tr>
<tr><th>Password:</th><td>${h.password('password')}</td></tr>
</table>
<input type="submit" value="Login" />
</form>
 
% if c.invalid_user:
<p>*** An invalid username or password was entered.</p>
% endif
 
<p>Don't have an account? Click <a href="/auth/register">here</a> to register.</p>
</body></html>

Notice that we used h.text() and h.password(). These helpers create our input boxes and will also display an error when we get into form validation. Make sure to import those functions in lib/helpers.py.

from webhelpers.html.tags import text, password

Now that we’ve created or login template, let’s implement our action auth/login in controllers/auth.py.

class AuthController(BaseController):
 
    def login(self):
        return render('login.mak')

Okay, let’s run the server and try to go to http://localhost:5000. We are redirect to our new login page.

$ paster serve --reload development.ini

Okay, now let’s implement our login action. This is under controllers/auth.py, and the action is login_post. The first thing we will do is add a formencode schema for our login form. This will make sure the username and/or password is not empty, and it will display an error accordingly. Here is what our new auth.py file might look like.

import formencode
 
class LoginForm(formencode.Schema):
    username = formencode.validators.String(not_empty=True)
    password = formencode.validators.String(not_empty=True)
 
class AuthController(BaseController):
 
    def login(self):
        return render('login.mak')
 
class AuthController(BaseController):
 
    def login(self):
        return render('login.mak')
 
    def login_post(self):
        try:
            form_result = LoginForm().to_python(request.POST)
            try:
                user = authenticate_user(form_result['username'], form_result['password'])
                session['user_id'] = user.id
                session.save()
                redirect_to('/')
            except InvalidUser:
                c.invalid_user = True
                return render('login.mak')
        except formencode.Invalid, err:
            html = render('login.mak')
            return formencode.htmlfill.render(html, errors=err.error_dict)

This login_post action first checks to see if the form is valid, if not, it will display the login form with the appropriate errors. If the form is valid, we attempt to call the function authenticate_user, which will return a valid User document if the login is successful, or raise an Exception if the login information is invalid. If the login is valid, we save the user.id in our session. If not, we set the invalid_user context variable and return the login page.

Okay, so we have our login_post action defined, but we don’t have an authenticate_user method yet. Let’s implement that. But before we do that, we need to know a little bit more about CouchDB views. A view is a way to query data from our database. A view is defined by implementing map and reduce functions.

Let’s start by adding a user view, that will let us query user data. To create the view, go to http://localhost:5984/_utils and select the userdemo database. Go to Create Document. We are going to create a design document which is where the views are defined. The document ID should be _design/user. This view is accessible via http://localhost:5984/userdemo/_view/user/VIEW_FUNC. To define our view functions, open the newly created _design/user document and add a field called views. For now, just put {} for the value. Hit save document.

Each view function defines a map and optionally a reduce function. This lets us limit and control what documents our view will return. For now, we just want a simple view that allows us to query the database and get a user by the username. Remember that CouchDB returns data with a key/value. The key we want to return is the username.

Let’s talk about map for a second. A map function is passed a CouchDB document, and then emits, or adds, key/value pairs. CouchDB uses javascript as the default view server. We will call our view function by_username, and it will return the username as the key and the user document as the value. Open your _design/user document and put this in for the views field.

{
    "by_username": {
        "map": "function(doc) { if(doc.type == 'user') emit(doc.username, doc); }"
    }
}

CouchDB should look like this:

Notice that we check doc.type. This is because all documents are stored under one namespace, so we need a way to differentiate between different documents. We do this by setting a type field for every user, with the value “user”. You can access this view by going here: http://localhost:5984/userdemo/_view/user/by_username.

Alright, now that we have our view defined, let’s implement the authenticate_user function. We also create a custom exception class that we use for an invalid user. We use hashlib to generate a sha256 hash of the user’s password and a random salt. gen_hash_password is used later for our registration functions, for creating a new salt and getting a hash. Add this above your controller in controllers/auth.py.

from userdemo.model import get_db, User
import hashlib
 
class InvalidUser(Exception):
    pass
 
def hash_password(password, salt):
    m = hashlib.sha256()
    m.update(password)
    m.update(salt)
    return m.hexdigest()
 
def gen_hash_password(password):
    import random
    letters = 'abcdefghijklmnopqrstuvwxyz0123456789'
    p = ''
    random.seed()
    for x in range(32):
        p += letters[random.randint(0, len(letters)-1)]
    return hash_password(password, p), p
 
def authenticate_user(username, password):
    result = User.view(get_db(), '_view/user/by_username', key=username)
    if len(result) == 0:
        raise InvalidUser('bad username')
 
    user = result.__iter__().next()
 
    # check password
    if not hash_password(password, user.salt) == user.password:
        raise InvalidUser('bad password')
 
    return user

Okay, now our site can login a user. Let’s get into registration. Create the following functions for our registration action in controllers/auth.py.

def register(self):
        return render('register.mak')
 
    def register_post(self):
        try:
            form_result = RegisterForm().to_python(request.POST)
            user = User()
            user.username = form_result['username']
            pwd, salt = gen_hash_password(form_result['password'])
            user.password = pwd
            user.salt = salt
            user.store(get_db())
 
            return 'You are registered. Click <a href="/auth/login">here</a> to login.'
        except formencode.Invalid, err:
            html = render('register.mak')
            return formencode.htmlfill.render(html, errors=err.error_dict)

These functions are similar to the login ones, except here we store a new user into the database, using the User schema document class that we defined in model/__init__.py. We also need to implement the RegisterForm, and also a custom validator, which checks to see if a username is taken.

def user_exists(username):
    result = User.view(get_db(), '_view/user/by_username', key=username)
    if len(result) == 0:
        return False
    return True
 
class UsernameValidator(formencode.validators.String):
    def validate_python(self, value, state):
        if user_exists(value):
            raise formencode.Invalid('Username already taken', value, state)
 
class RegisterForm(formencode.Schema):
    username = UsernameValidator(not_empty=True)
    password = formencode.validators.String(not_empty=True)

And that’s it! You can now register an account, login, and view the protected page(with the require_login decorator). Now you just need to add require_login to any function that is only for logged in users. In the next tutorial, I will go into some more advanced CouchDB topics and some cool map/reduce views.

You can download the pylons userdemo project here.

Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API.

Normally a Pylons web application will use some sort of RDBMS for storing data and persistence — such as MySQL or PostgreSQL. I’ve decided to go a different route and integrate Pylons with CouchDB. Right now there is a Python library, couchdb-python, that will help us communicate with our CouchDB’s HTTP/JSON api.

For this article, I will assume you have Pylons installed and you are somewhat familiar with it. We are going to create a simple page counter application.

CouchDB is written in Erlang, a functional programming language with high concurrency. Let’s install Erlang.

$ wget http://erlang.org/download/otp_src_R12B-5.tar.gz
$ tar -xvzf otp_src_R12B-5.tar.gz
$ cd otp_src_R12B-5
$ ./configure
$ sudo make && make install

You should now have erlang installed. If you type erl and hit enter, you should see something like this.

$ erl
Erlang (BEAM) emulator version 5.6.3 [source] [smp:2] [async-threads:0] [kernel-poll:false]
 
Eshell V5.6.3  (abort with ^G)
1>

Now let’s install CouchDB. The latest version can be found on this page.

$ wget http://download.filehat.com/apache/incubator/couchdb/0.8.1-incubating/apache-couchdb-0.8.1-incubating.tar.gz
$ tar -xvzf apache-couchdb-0.8.1-incubating.tar.gz
$ cd apache-couchdb-0.8.1-incubating
$ ./configure
$ sudo make && make install

Okay. CouchDB should now be installed. Let’s start it up.

$ couchdb
Apache CouchDB 0.9.0a721128-incubating (LogLevel=info) is starting.
Apache CouchDB has started. Time to relax.

Alright, we are almost there. Let’s install the couchdb-python library.

easy_install:

$ sudo easy_install CouchDB==0.5

or the latest development version:

$ svn checkout http://couchdb-python.googlecode.com/svn/trunk/ couchdb-python-read-only
$ cd couchdb-python-read-only
$ sudo python setup.py install

Now that we’ve got CouchDB all setup and running, let’s open up the web interface and create a database. CouchDB has a great web interface, located at http://localhost:5984/_utils/.

When you are at the web interface, go to create database, and let’s name it tutorial. After you have created the new database, you are redirected and are shown an empty database with no documents.

We now have created our CouchDB database and we are ready to start adding new documents.

Let’s create a new Pylons project named tut1, delete the default index.html, and a create a controller called main.

$ paster create -t pylons tut1
$ cd tut1
$ rm tut1/public/index.html
$ paster controller main

For this tutorial, we aren’t really going to go into too much detail. We are just going to make one page with a simple page counter. The next tutorial will be about some more advanced topics.

Okay, open up your config/routing.py file so we can add a route to our index page in the main controller.

map.connect('/', controller='main', action='index')
map.connect('/{controller}/{action}')
map.connect('/{controller}/{action}/{id}')

Now let’s open up the file controllers/main.py. Let’s add an import for the couchdb-python module.

import couchdb.client

For the counter to work, we are going to just create a simple document that will store the visits.

Here is what our method for index will look like.

    def index(self):
        db = couchdb.client.Database('http://localhost:5984/tutorial')
 
        # if counter does not exist, let's initialize it
        if not 'counter' in db:
            db['counter'] = {'visits': 1}
            return 'You are the first user!'
        else:
            counter = db['counter']
            counter['visits'] = counter['visits'] + 1
 
            # save
            db['counter'] = counter
 
            return 'You are user #%d.' % (counter['visits'])

First, a server object is created, and then the database is selected. After, we check to see if the counter document exists, and if it doesn’t, we initialize it. CouchDB stores a documents as a dictionary. A document can have as many key/value pairs as it wants. For this example, we just have one key, visits, which stores an integer.

Now that our index method is done, let’s run our pylons application.

$ paster serve --reload development.ini

Open up the site in a web browser, at http://localhost:5000.

You should see the following on your first visit.

You are the first user!

And on subsequent visits…

You are user #2.

You are user #3.

You have successfully created a CouchDB powered page counter! Now, this isn’t meant to be a production page counter, but it should give you a basic example of how CouchDB stores and retrieves documents.

In the next tutorial, we will integrate CouchDB with Pylons more, use Forms, CouchDB-Python’s Schemas, and more.

Note: The first time your page loads it will not display if the page is valid or not(It is validated AFTER the first response).

Download Link: http://www.tech9computers.com/W3ValidatorMiddleware-0.1-py2.5.egg

Source Download(gzipped): http://www.tech9computers.com/w3validator.tar.gz

Install

wget http://www.tech9computers.com/W3ValidatorMiddleware-0.1-py2.5.egg
sudo easy_install W3ValidatorMiddleware-0.1-py2.5.egg

Configuration

config/middleware.py

At the top of your middleware.py import the following

1

from w3validator import ValidatorMiddleware

Then right below CUSTOM MIDDLEWARE HERE

1
2

    # CUSTOM MIDDLEWARE HERE (filtered by error handling middlewares)
    app = ValidatorMiddleware(app)

lib/helpers.py

At the top you need to import cache_is_valid_xhtml()

1

from w3validator import cache_is_valid_xhtml

I then created the following method that I call from my templates to return a string saying if the page is valid or not.

def xhtml_str():
    valid = cache_is_valid_xhtml()
 
    if valid == True:
        return '<em>Valid</em> XHTML 1.0'
    elif valid == False:
        return '<em>NOT</em> Valid XHTML 1.0'
    else:
        # hasn't been validated yet
        return 'XHTML 1.0'

Template Configuration

If you have a base template, you might want to the footer of every page to display if it is valid (X)HTML or not.

Mako

1
2
3
4
5
6
7
8

<html>
    <body>
        <h1>My XHTML Site</h1>
    <div id="footer">
        ${h.xhtml_str()}
    </div>
    </body>
</html>

That’s all you need to do to add validation support to your site. You can check the middleware out in action on my site at http://blackberrytracker.com. Scroll to the bottom of the page to see where I have the validation result.

Django is a great web framework and I think it is perfect for people that want to build news/content sites. Everything in Django works very well together which makes it a one-stop shop.

Pylons is awesome too but compared to Django is lacking in a lot of documentation. You need to look in a lot of different places to learn all the aspects that Pylons has to offer. I will say that Genshi + SQLAlchemy + Pylons + Routes is very very powerful.

I’m going to be moving on to TurboGears now to see what is has different. I’ve read that the documentation is more complete but for the most part the components in each framework are pretty much the same(or can be).

Here it goes.