chris moos’s blog

I was reading a thread over at pylons-discuss about worker threads in pylons. Worker threads are useful if you need to execute a long running task, but want to return to the user immediately. You could run a new thread per request, but if you have many requests, it is probably better to queue things up.

To do this, you start a thread and use python’s Queue for managing the tasks. Here is a very basic implementation:

config/environment.py:

# PYLONS IMPORTS
 
from myapp.lib.myworker import start_myworker
 
def load_environment(global_conf, app_conf):
    """Configure the Pylons environment via the ``pylons.config``
    object
    """
 
    # PYLONS STUFF GOES HERE
 
    # start worker
    start_myworker()

lib/myworker.py:

import Queue
import threading
 
worker_q = Queue.Queue()
 
class MyWorkerThread(threading.Thread):
    def run(self):
        print 'Worker thread is running.'
 
        while True:
            msg = worker_q.get()
            try:
                # do a long running task...
                print 'We got %s, do something with it!' % (msg)
            except Exception, e:
                print 'Unable to process in worker thread: ' + str(e)
            worker_q.task_done()                
 
def start_myworker():
    worker = MyWorkerThread()
    worker.start()

And in your controller….

from myapp.lib.myworker import worker_q
 
class WorkerController(BaseController):    
    def do_task(self):
      worker_q.put({'id': generate_some_unique_id(), 'msg': 'your data goes here'})

In the previous tutorial, we learned how to get CouchDB and Pylons up and running, as well as create a simple page counter. Now we are going to implement a simple user authentication system. This tutorial will teach you how to use formencode to validate forms and CouchDB to store our user data.

Let’s start by creating a new pylons project and some controllers.

$ paster create -t pylons userdemo
$ cd userdemo
$ paster controller main
$ paster controller auth

Also, delete public/index.html.

For our controller main, we are going to add one action called index. This will be the main page for the site and will only be accessible for logged in users. If a user is not logged in, they will be redirected to the login page. Let’s add some routes for the main page, login, logout, and registration. Open up config/routing.py.

    map.connect('/', controller='main', action='index')
    map.connect('/auth/login', controller='auth', action='login', conditions=dict(method=['GET']))
    map.connect('/auth/login', controller='auth', action='login_post', conditions=dict(method=['POST']))
    map.connect('/auth/logout', controller='auth', action='logout')
    map.connect('/auth/register', controller='auth', action='register', conditions=dict(method=['GET']))
    map.connect('/auth/register', controller='auth', action='register_post', conditions=dict(method=['POST']))

For checking to see if a user is logged in, we will store a user_id in the session. Let’s make a decorator that we can add to our index action that will check to see if a user is logged in. If the user isn’t logged in, it will redirect to the login page. Your controllers/main.py should look have this:

from decorator import decorator
 
def require_login(func, *args, **kwargs):
    """ Checks to see if user_id is in session """
    if not 'user_id' in session:
        redirect_to('/auth/login')
    return func(*args, **kwargs)
require_login = decorator(require_login)
 
class MainController(BaseController):
 
    @require_login
    def index(self):
        return 'You are logged in! Click <a href="/auth/logout">here</a> to logout.'

Continue reading ‘CouchDB and Pylons: User Registration and Login’ »

CouchDB is very cool(it’s built on erlang), and with Pylons, it is even cooler.

Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API.

Normally a Pylons web application will use some sort of RDBMS for storing data and persistence — such as MySQL or PostgreSQL. I’ve decided to go a different route and integrate Pylons with CouchDB. Right now there is a Python library, couchdb-python, that will help us communicate with our CouchDB’s HTTP/JSON api.

For this article, I will assume you have Pylons installed and you are somewhat familiar with it. We are going to create a simple page counter application.

CouchDB is written in Erlang, a functional programming language with high concurrency. Let’s install Erlang.

$ wget http://erlang.org/download/otp_src_R12B-5.tar.gz
$ tar -xvzf otp_src_R12B-5.tar.gz
$ cd otp_src_R12B-5
$ ./configure
$ sudo make && make install

You should now have erlang installed. If you type erl and hit enter, you should see something like this.

$ erl
Erlang (BEAM) emulator version 5.6.3 [source] [smp:2] [async-threads:0] [kernel-poll:false]
 
Eshell V5.6.3  (abort with ^G)
1>

Continue reading ‘CouchDB and Pylons: Getting Started’ »

This middleware will use the W3 Validator Service to check if your page is valid HTML. It will only check each URI once so that it doesn’t slow down your site by validating every page load. It does this by placing the result of each URI into a memory cache(using Beaker). You can then call cache_is_valid_xhtml() in your template to determine if the current page is valid.

Note: The first time your page loads it will not display if the page is valid or not(It is validated AFTER the first response).

Download Link: http://www.tech9computers.com/W3ValidatorMiddleware-0.1-py2.5.egg

Source Download(gzipped): http://www.tech9computers.com/w3validator.tar.gz

Install

wget http://www.tech9computers.com/W3ValidatorMiddleware-0.1-py2.5.egg
sudo easy_install W3ValidatorMiddleware-0.1-py2.5.egg

Configuration

config/middleware.py

At the top of your middleware.py import the following

1

from w3validator import ValidatorMiddleware

Then right below CUSTOM MIDDLEWARE HERE

1
2

    # CUSTOM MIDDLEWARE HERE (filtered by error handling middlewares)
    app = ValidatorMiddleware(app)

lib/helpers.py

At the top you need to import cache_is_valid_xhtml()

1

from w3validator import cache_is_valid_xhtml

I then created the following method that I call from my templates to return a string saying if the page is valid or not.

def xhtml_str():
    valid = cache_is_valid_xhtml()
 
    if valid == True:
        return '<em>Valid</em> XHTML 1.0'
    elif valid == False:
        return '<em>NOT</em> Valid XHTML 1.0'
    else:
        # hasn't been validated yet
        return 'XHTML 1.0'

Template Configuration

If you have a base template, you might want to the footer of every page to display if it is valid (X)HTML or not.

Mako

1
2
3
4
5
6
7
8

<html>
    <body>
        <h1>My XHTML Site</h1>
    <div id="footer">
        ${h.xhtml_str()}
    </div>
    </body>
</html>

That’s all you need to do to add validation support to your site. You can check the middleware out in action on my site at http://blackberrytracker.com. Scroll to the bottom of the page to see where I have the validation result.

On my trial to find the best python web framework(all around), I’ve just finished learning Pylons and experiencing everything it has to offer. Genshi is amazing and kills any text template engine(Django, Mako). SQLAlchemy is much different than Django’s ORM and has its advantages and disadvantages. Django’s ORM feels more natural and is much easier to learn at first. I believe that SQLAlchemy is much more powerful but the learning curve is much bigger.

Django is a great web framework and I think it is perfect for people that want to build news/content sites. Everything in Django works very well together which makes it a one-stop shop.

Pylons is awesome too but compared to Django is lacking in a lot of documentation. You need to look in a lot of different places to learn all the aspects that Pylons has to offer. I will say that Genshi + SQLAlchemy + Pylons + Routes is very very powerful.

I’m going to be moving on to TurboGears now to see what is has different. I’ve read that the documentation is more complete but for the most part the components in each framework are pretty much the same(or can be).

Here it goes.