chris moos's blog

Getting started with Scala using SBT

Chris Moos — Thu, 11 Feb 2010 02:13:03 +0000

One of my biggest gripes with Java (and all the languages that run on the JVM) is getting my project setup and building it. Maven is not my favorite, and ant..well..I don’t like it either. Fortunately, if you want to start a new project in Scala, there is a great build tool available that takes a lot of the pain out of project management and building – SBT, simple-build-tool.

sbt is a simple build tool for Scala projects that aims to do the basics well. It requires Java 1.5 or later.

Installing SBT

I’m using Mac OS X, but the following instructions should be pretty much the same on any Unix based OS.

You can find the latest version of SBT here.

cd ~
wget http://simple-build-tool.googlecode.com/files/sbt-launcher-0.5.6.jar
sudo mv sbt-launcher-0.5.6.jar /usr/local/bin/sbt-launcher.jar
echo "java -Xmx512M -jar /usr/local/bin/sbt-launcher.jar \"\$@\"" | sudo tee /usr/local/bin/sbt
sudo chmod +x /usr/local/bin/sbt

This will install the SBT jar and create a script called sbt that will allow you to run the sbt jar.

Just type sbt and press enter, and you know have access to sbt.

$ sbt
Project does not exist, create new project? (y/N/s) : n

Creating a new Scala project

Now we will create a Hello World Scala project with SBT.

mkdir hello_scala
cd hello_scala

Running the sbt command in a directory where there is no project will prompt you to create one.

sbt

Project does not exist, create new project? (y/N/s) : y
Name: Hello, Scala!
Organization []: 
Version [1.0]: 
Scala version [2.7.7]: 
sbt version [0.5.6]: 
:: retrieving :: sbt#boot
	confs: [default]
	2 artifacts copied, 0 already retrieved (9911kB/72ms)
:: retrieving :: sbt#boot
	confs: [default]
	3 artifacts copied, 0 already retrieved (3409kB/15ms)
[success] Successfully initialized directory structure.
[info] Building project Hello, Scala! 1.0 using sbt.DefaultProject
[info]    with sbt 0.5.6 and Scala 2.7.7
[success] Build completed successfully.
[info] 
[info] Total build time: 0 s

Awesome. It handles all the Scala dependencies for us! Now let’s create a file that contains our Hello, Scala example.

Below is the directory structure of an SBT project.

$ ls
lib	project	src

Creating our HelloScala sources and running

Now we are going to create our main file, HelloScala.scala

src/main/scala/HelloScala.scala

object HelloScala {
  def main(args: Array[String]) {
    println("Hello, Scala!")
  }
}

And now we can build and run it by just issuing the following:

sbt run

And the output:

[info] Building project Hello, Scala! 1.0 using sbt.DefaultProject
[info]    with sbt 0.5.6 and Scala 2.7.7
[info] 
[info] == compile ==
[info]   Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling main sources...
[info] Nothing to compile.
[info]   Post-analysis: 2 classes.
[info] == compile ==
[info] 
[info] == copy-resources ==
[info] == copy-resources ==
[info] 
[info] == run ==
[info] Running HelloScala ...
Hello, Scala!
[info] == run ==
[success] Successful.
[info] 
[info] Total time: 0 s
[success] Build completed successfully.
[info] 
[info] Total build time: 1 s

And that’s it. Setting up a new Scala project with SBT is painless. In the next part I will talk about managing dependencies and how SBT makes this also very easy.

You can read a lot more about SBT by checking out their wiki.

AjaxTask – a rails plugin for managing background tasks

Chris Moos — Wed, 10 Feb 2010 06:14:27 +0000

SOAP, Background Tasks, and AJAX

Recently in Rails I’ve been interacting with various SOAP services and running them in the background with Workling. I needed to relay the SOAP response to the client’s web browser, so I decided to use AJAX to poll the status of my background tasks.

This is great if you have < 30 second background tasks running, but don't want to block a user (and a request).

The Solution

I created a Rails plugin, called AjaxTask, that has two components:

Methods to use in your controller to define a task handler and create tasks
Javascript library to manage the AJAX between the browser and the handler.

GitHub Link: http://github.com/chrismoos/ajaxtask

In a nutshell, the client initiates a task, the handler responds with a task ID, and the client polls at a user defined interval until the task has finished, or has an error.

The plugin takes the pain out of implementing the handler, as well as the Javascript. All you have to do is run code for your task, and periodically update the status.

I am using Workling to run my background tasks, as well as maintain the status using Workling’s return store.

Okay, enough with the intro, here is the example.

Example

Controller/Routes

The first thing to do is define the handler. This instructs the AjaxTask plugin to create a handler that will respond to ajax requests, as well as dispatch to your actual tasks. The only parameter to ajaxtask_handler is a symbol, which MUST be identical to a named route. This is how a URL gets from Rails to the plugin.

routes.rb:

map.ajaxtask_demo '/ajaxtask/handler/:task', :controller => :demo, :action => :ajaxtask_demo

demo_controller.rb:

ajaxtask_handler :ajaxtask_demo

Now we will define a task:

demo_controller.rb:

ajaxtask :mytask

This tells the plugin to respond to a task named mytask.

By doing this, we must implement two methods in our controller.

mytask_start is called when a browser starts a new task. You should probably fire off your background task in this method.

mytask_start should return a unique ID for the task. By using Workling and calling .async, a unique ID is returned.

def mytask_start
    return MyWorklingWorker.async_mytask
end
 
def mytask_status(uid)
    Workling.return.get(uid)
end

The Worker

The worker is the meat of our background task. In this we will do something that might take a while, and also update the status.

my_workling_worker.rb:

class MyWorklingWorker < Workling::Base
  def mytask(options)
    Workling.return.set(options[:uid], {:pending => 'i am just starting...wait up!'})
    begin
      # your long running task goes here
			sleep 10
    rescue => e
      Workling.return.set(options[:uid], {:error => e.to_s})
      return
    end
    Workling.return.set(options[:uid], {:done => 'i finished!})
  end
end

The important things to note here are what we set the return to. AjaxTask recognizes the following:

:error
:pending
:done

They should be pretty self explanatory. Now let’s see what the client side looks like.

The Client

For the client, we will be interacting with the AjaxTask javascript library. Make sure you copy the ajaxtask.js file to your javascripts directory, and include it in your page. The following will copy the javascript for you:

cd vendor/plugins/ajaxtask
rake ajax_task_js

Here is an example of what an HTML page that uses AjaxTask:

>
>
	<%= javascript_include_tag 'ajaxtask.js' %>
>
>
>
function mytaskHandler() {
	$(this).bind('onTaskError', onTaskError);
	$(this).bind('onTaskFinished', onTaskFinished);
	$(this).bind('onTaskPending', onTaskPending);
 
	function onTaskError(event, error) {
		alert('error: ' + error);
	}
 
	function onTaskPending(event, data) {
		alert("pending: " + data);
	}
 
	function onTaskFinished(event, data) {
		alert("finished: " + data);
	}
}
 
$(document).ready(function() {	
	var myTask = new AjaxTask({
		url: "<%= ajaxtask_demo_url :task => :mytask %>",
		handler: new mytaskHandler(),
		taskStatusDiv: $("#taskStatus"),
		taskStatusLoadingMsg: 'Please wait while my task runs...',
		taskStatusLoadingImg: '/images/smallactivity.gif',
		taskStatusErrorMsg: 'Oops...something bad happened.'
	});
	myTask.start();
});
>
 id="taskStatus">>
 
>
>

Looking at the above client code, you can see how easy it is to present a background task’s processing to a user.

That does it for now, I’ll try to document and post more soon abou AjaxTask.

MySQL and partitioning tables with millions of rows

Chris Moos — Sun, 31 Jan 2010 19:43:58 +0000

The Problem

I’ve been running a mobile GPS tracking service, MoosTrax (formerly BlackBerry Tracker), for a few years and have encountered a large amount of data in the process.

A user’s phone sends its location to the server and it is stored in a MySQL database. Each “location” entry is stored as a single row in a table.

Right now there are approximately 12 million rows in the location table, and things are getting slow now, as a full table scan can take ~3-4 minutes on my limited hardware. This means that if a user is pulling a location from history it could potentially block all other users (as the table is locked) access to the site until the query is complete.

Partitioning

Partitioning allows you to store parts of your table in their own logical space. With partitioning, you want to divide up your rows based on how you access them. If you partition your rows and you are still hitting all the partitions, it does you no good. The goal is that when you query, you will only have to look at a subset of the data to get a result, and not the whole table.

There are various ways in MySQL to partition a database, such as:

RANGE – rows are partitioned based on the range of a column (i.e date, 2006-2007, 2007-20008, etc,.)
HASH – hashes a column and depending on the result of the hash, has a different partition
LIST, KEY

Choosing the partition type is important, so I looked at how my application looks up a user’s location.

Getting a user’s current location

Location.find(:all, :conditions => {:device_id => @device.id}, :order => "date_added desc", :limit => 6)

Getting a users’s location history

Location.find(:all, :conditions => {:date_added => startdate.utc..enddate.utc, :device_id => @device.id}, :order => "date_added desc", :limit => 500)

At first, I thought about RANGE partitioning by date, and while I am using the date in my queries, it is very common for a query to have a very large date range, and that means it could easily span all partitions.

After a second look, it seemed that device_id might be the best, using the HASH partitioning type.

This means that all the locations would be partitioned equally by their device_id. This is great because MoosTrax is only looking at one device at a time, history or live tracking, and doesn’t aggregate the locations across devices or users.

Preparing to partition

First, to partition a table the column you want to partition by must be part of the primary key. I only had “id” in my primary key, so I modified it to include my partitioning column, device_id.

Drop the Primary Key

ALTER TABLE location DROP PRIMARY KEY

Partition the table

Now we are going to add our new primary key, and tell MySQL to partition, with HASH, by device_id. We also specify the option, partitions, to tell MySQL how many partitions we want it to use. I believe the limit is 1024.

ALTER TABLE location 
ADD PRIMARY KEY (id, device_id)
partition BY HASH(device_id)
partitions 200

FYI: Running the above may take a while depending on the size of your table.

Does it work?

MySQL has a command that we can run, explain partitions, that will let us specify a query, and MySQL will tell us if and how it is using partitioning to get the result.

Because we partitioned by device_id, let’s try a simple select with device_id in the where clause.

mysql> explain partitions select * from location where device_id = 1;
+----+-------------+----------+------------+------+---------------+-----------+---------+-------+------+-------+
| id | select_type | table    | partitions | type | possible_keys | key       | key_len | ref   | rows | Extra |
+----+-------------+----------+------------+------+---------------+-----------+---------+-------+------+-------+
| 1  | SIMPLE      | location | p1         | ref  | device_id     | device_id | 4       | const | 1    |       |
+----+-------------+----------+------------+------+---------------+-----------+---------+-------+------+-------+
1 rows in set (0.14 sec)
 
mysql>

If you look at the result of the explain, you can see that MySQL only needs to use partition p1 to find our result..this is great! There are way less rows in the partition than in the whole table.

Now let’s try another query, that won’t use our partitioning column.

 
mysql> explain partitions select * from location where date_added > '2009-10-10';
+----+-------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+---------------+------+---------+------+---------+-------------+
| id | select_type | table    | partitions                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | type | possible_keys | key  | key_len | ref  | rows    | Extra       |
+----+-------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+---------------+------+---------+------+---------+-------------+
| 1  | SIMPLE      | location | p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20,p21,p22,p23,p24,p25,p26,p27,p28,p29,p30,p31,p32,p33,p34,p35,p36,p37,p38,p39,p40,p41,p42,p43,p44,p45,p46,p47,p48,p49,p50,p51,p52,p53,p54,p55,p56,p57,p58,p59,p60,p61,p62,p63,p64,p65,p66,p67,p68,p69,p70,p71,p72,p73,p74,p75,p76,p77,p78,p79,p80,p81,p82,p83,p84,p85,p86,p87,p88,p89,p90,p91,p92,p93,p94,p95,p96,p97,p98,p99,p100,p101,p102,p103,p104,p105,p106,p107,p108,p109,p110,p111,p112,p113,p114,p115,p116,p117,p118,p119,p120,p121,p122,p123,p124,p125,p126,p127,p128,p129,p130,p131,p132,p133,p134,p135,p136,p137,p138,p139,p140,p141,p142,p143,p144,p145,p146,p147,p148,p149,p150,p151,p152,p153,p154,p155,p156,p157,p158,p159,p160,p161,p162,p163,p164,p165,p166,p167,p168,p169,p170,p171,p172,p173,p174,p175,p176,p177,p178,p179,p180,p181,p182,p183,p184,p185,p186,p187,p188,p189,p190,p191,p192,p193,p194,p195,p196,p197,p198,p199 | ALL  | date_added    | NULL | NULL    | NULL | 12641367 | Using where |
+----+-------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+---------------+------+---------+------+---------+-------------+
1 rows in set (1.81 sec)
 
mysql>

As you can see, MySQL would need to go through all 200 partitions to get the result. Fortunately, MoosTrax doesn’t use a query like that, as the device_id is always available. Therefore, if I am searching by date, I will also specify the device_id as well, so that MySQL will use the partition.

mysql> explain partitions select * from location where date_added > '2009-10-10' and device_id = 1;
+----+-------------+----------+------------+------+----------------------+-----------+---------+-------+------+-------------+
| id | select_type | table    | partitions | type | possible_keys        | key       | key_len | ref   | rows | Extra       |
+----+-------------+----------+------------+------+----------------------+-----------+---------+-------+------+-------------+
| 1  | SIMPLE      | location | p1         | ref  | device_id,date_added | device_id | 4       | const | 1    | Using where |
+----+-------------+----------+------------+------+----------------------+-----------+---------+-------+------+-------------+
1 rows in set (0.11 sec)
 
mysql>

That’s better. Now its using our partitions correctly.

As long as you always use your partitioning column in your query, you will be able to take advantage of the partitioning.

The Result

After switching to partitioning, many queries are running much much faster than before. I couldn’t be happier.

If you want to read more about MySQL partitioning, check out the manual.

Why I didn’t like Java 5 years ago, and why I don’t like it now

Chris Moos — Tue, 08 Dec 2009 01:35:34 +0000

Then

I started out programming in C, which taught me a lot about the fundamentals of computer science. I learned about types, memory management, functions, and logic. As I began to evaluate other programming languages to try out, I of course ended up trying Java. My first impression of it was how heavy it felt. Of course, this was when 4GB of memory wasn’t standard in a desktop, and memory allocations and processing power — were still relatively precious.

I remember trying out Swing and that only made me more disgusted with Java, as a Swing application felt horribly slow.

Java users don’t have to worry about memory management (technically), as the garbage collection system takes care of it for the user. I think this was a huge benefit for novice developers, because dealing with memory management definitely isn’t fun — and usually presents issues if not done properly.

The next thing I tried in Java was creating a web application. I bought a book on J2EE and as I began learning the ins and outs, I began to hate it with a passion. The amount of configuration and boiler plate code to get something simple up and running, was a huge turn off to me. I was disgusted with the concept of EJBs and all the various patterns in J2EE.

After a short while, J2EE was gone with the wind for me. I moved on to scripting languages, such as the notoriously shitty PHP, which was still in my opinion, more practical than Java…but I wouldn’t settle on a good web framework and language until Python and Ruby really caught my eye.

Now

Flash forward to today…and I’m still not liking Java.

It is still plagued with lots of configuration, descriptors, assembly, and boiler plate code. And now that memory is relatively cheap and available, Java still eats it like a fat boy eating at McDonalds. PermGen errors, anyone? The JVM has moved forward a lot in the past years, but its still a memory hog, and I feel like it abstracts so much low level coding that developers tend to not pay attention to the performance of a system — just throw more hardware at the JVM.

My productivity in Java is much lower than most other languages — even C. When building enterprise software in Java the complexity of getting it setup and going seems like too much at times.

DAO’s, interfaces, implementations, proxies — its just boring to me.

Ever used a BlackBerry? It feels slow to me…and I bet if it was coded in C it would be a lot snappier…same goes for Android. I’m currently using an iPhone now and it definitely feels the most repsonsive out of all 3.

What do you think about Java? Any recommendations on feeling more productive and not slowed down?

libactor now at google code

Chris Moos — Sat, 31 Oct 2009 06:52:55 +0000

libactor is now available on google code. Check it out: http://code.google.com/p/libactor/

If you have any problems or ideas, post them there!

Chris

libactor – a C library based on the Actor model

Chris Moos — Thu, 29 Oct 2009 06:38:38 +0000

Recently I’ve been really interested in functional, concurrent programming languages, such as Erlang. This prompted me to have some fun in C, and try to implement a simple library that is based on the Actor Model.

Right now it is usable, although it may not be ready for production. It uses pthreads and the library handles all of the threading issues, so you don’t have to worry about any of it at all.

In a future version I plan to add more sandboxing to the actors, so that when one actor crashes, they don’t all go down!

Right now it supports the following:

Spawn actor
Sending messages
Broadcast message
Actor memory management convenience functions(when an actor dies, the memory is freed!)

Here is what a simple ping/pong example looks like:

#include 
#include 
 
void *pong_func(void *args) {
        actor_msg_t *msg;
 
        while(1) {
                msg = actor_receive();
                if(msg->type == PING_MSG) {
                        printf("PING! ");
                        actor_reply_msg(msg, PONG_MSG, NULL, 0);
                }
                arelease(msg);
        }
}
 
void *ping_func(void *args) {
        actor_msg_t *msg;
        actor_id aid = spawn_actor(pong_func, NULL);
        while(1) {
                actor_send_msg(aid, PING_MSG, NULL, 0);
                msg = actor_receive();
                if(msg->type == PONG_MSG) printf("PONG!\n");
                arelease(msg);
                sleep(5);
        }
}
 
 
void *main_func(void *args) {
        struct actor_main *main = (struct actor_main*)args;
        int x;
 
        /* Accessing the arguments passed to the application */
        printf("Number of arguments: %d\n", main->argc);
        for(x = 0; x < main->argc; x++) printf("Argument: %s\n", main->argv[x]);
 
        /* PING/PONG example */
        spawn_actor(ping_func, NULL);
}
 
DECLARE_ACTOR_MAIN(main_func)

There is a more detailed example in the source distribution, located in the examples/ directory.

You can get the source here. Please, let me know what you think about it, it was really fun to write!

Documentation is available here: http://chrismoos.com/libactordocs/

Colloquy for iPhone — Push Notification Server

Chris Moos — Fri, 23 Oct 2009 00:04:42 +0000

Colloquy is a great IRC client for Mac, as well as the iPhone. Currently, to run Colloquy on the iPhone you must purchase the mobile version from the App Store. The other option is to download the source and build it yourself, but you must be a registered iPhone developer to run it on an actual iPhone. Also, if you intend to use push notifications (via the Colloquy ZNC plugin), you must have a push certificate as well.

Getting Started

I am going to assume that you have znc installed, and you know how to use it. Next, we need to install the Colloquy ZNC module, so that Colloquy can communicate with ZNC (and allow us to have push functionality). See this page for complete instructions.

The module for ZNC accepts commands from the Colloquy client, notably information on push notifications(device token, etc,.).

Modifying the Colloquy Mobile Source

Next, we need to download and build the source. Once you have downloaded the source and loaded up the Mobile Colloquy project in Xcode, we need to make a few modifications.

Certificates

To run any application on an actual iPhone device, it must be signed. Login to your Developer portal on Apple’s site and generate a certificate (and make sure to enable push). Download the certificate and install it. Next, assign that certificate to the Colloquy Mobile Project. Project…Edit Active Target. Make sure that under the Properties tab, you have the same Identifier name that you used when generating the certificate. Go to Build, and then Code Signing Identity and select the certificate.

Modifying the source

Next, find the CQConnectionsController.m file (Controllers/Connections) and change the following line from:

[self sendRawMessage:@"PUSH service colloquy.mobi 7906"];

and change it to:

[self sendRawMessage:@"PUSH service 127.0.0.1 7906"];

This will make the client tell the Colloquy ZNC module to send push notifications to our custom server.

Build the project and put it onto your device.

Push Server

I wrote a simple python script that waits on port 7906, receives a connection from the Colloquy ZNC module, and then pushes a notification to your phone (via Apple’s Push Service).

The script requires your push certificate and private key file so it can authenticate to Apple’s push service.

Export the push certificate and private key as a .p12 from Keychain Access. After that, we have to convert it to PEM format:

openssl pkcs12 -in key.p12 -out certs.pem -nodes

where key.p12 is the file exported from Keychain access.

After that, you may need to put that certs.pem file on your server that you are running ZNC on.

Installing the Colloquy Push Server

Run the following to install the Colloquy Push Server:

sudo easy_install http://www.tech9computers.com/ColloquyPushServer-0.1.tar.gz

Next, run the command colloquypushsrv.py to make sure installation was successful.

The usage looks like this:

usage: /usr/local/bin/colloquypushsrv.py [-p listen_port] [--sandbox] [-b] certs_file
	Command Summary:
		certs_file	        PEM encoded Apple Push certificate and private key file(absolute path)
		-p listen_port	Default Listen Port: 7906	
		--sandbox 	Connect to Sandbox Service
		-b		        Run in background

Now, let’s start the push server in the foreground. Make sure to specify the full path to the pem file we created.

colloquypushsrv.py ~/certs.pem

Conclusion

Finally, connect to your ZNC server with the custom built Colloquy Mobile. Your iPhone should register with the ZNC module. After, exit the Colloquy Mobile application and pull up Colloquy (or any other IRC client) on your desktop. Send yourself a message, and see if you get the push notification!

I realize this process may not be completely clear, so if you have any questions, I would be happy to walk you through the process.

Also, I just wrote this up today, so it may not be production ready .

Enjoy!

Writing a FreeSWITCH Caller ID spoofing interface

Chris Moos — Sat, 17 Oct 2009 00:06:21 +0000

Lately I have been really interested in voice over IP, I think there is a lot of power in it. I was using Asterisk, but I’ve moved onto a much better and more powerful solution, FreeSWITCH. I was playing with it recently and decided a Caller ID spoofing interface would be fun.

When the following script is ran, it will prompt you for an access code, and if it is correct, it will let you specify a number to spoof, and a number to call. Its fun . Also, it can send an email whenever a spoofed call is made (if you want to track your friends usage, for example).

To use the following script, put this in your dialplan: (I’ve put spoof.py in the fspy package.)

 action="python" data="fspy.spoof"/>

spoof.py

import os,sys
from freeswitch import *
from datetime import datetime
from time import mktime
import httplib
import urllib
import random
import smtplib
 
def get_number(session,invalid,num=10):
	digits = session.getDigits(num, "#", 6000)
	if digits == '':
		if invalid == 3:
			session.speak('You have entered too many in valid entries. Goodbye.')
			session.hangup()
		else:
			session.speak('Invalid entry. Please try again.')
			get_number(session,invalid+1)
	else:
		return digits
 
def handler(session, args):
	session.answer()
	session.set_tts_parms('cepstral', 'Allison')
 
	session.speak('Please enter the 4 digit access code.')
	code = get_number(session,2,num=4)
	if(code == None or code != '6969'):
		session.speak('Invalid access code, goodbye!')
		session.hangup()
		return
	session.speak('Please enter the number you would like to spoof.')
	spoof_number = get_number(session, 0)
	session.speak('Please enter the number you would like to call.')
	dest_number = get_number(session, 0)
	session.setVariable("effective_caller_id_number", spoof_number)
 
	session.speak('Calling...please wait')
	server = smtplib.SMTP('smtp.mydomain.com')
 
	fromaddr = 'freeswitch@mydomain.com'
	toaddrs = ['someaddr@mydomain.com']
	subject = "A spoofed call was made"
	msg = ("From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n" % (fromaddr, ", ".join(toaddrs), subject))
	d = datetime.now().strftime('%A, %B %d, %Y %I:%M%p')
	msgContents = "Date: %s\nCaller ID: %s\n\nDestination Number: %s\nSpoofed Number: %s\n\n\nSincerely, \nFreeSwitch Spoofer" % (d, session.getVariable("caller_id_number"), dest_number, spoof_number)
 
	server.sendmail(fromaddr, toaddrs, msg+msgContents)
	session.execute('transfer', dest_number)

Building an Erlang chat server with Comet – Part 3

Chris Moos — Fri, 16 Oct 2009 23:32:12 +0000

You can see a live demo of the following tutorial here.

Overview

In the last part, we implemented the first version of our chat server. Now that the framework is down, we can start adding some more features. You can find the source for this tutorial here

In this part, we will add the following features:

Rate Limiting for Messages
Admin Login
Kick/Ban users
Limiting connections per host

Rate Limiting

This feature will *help* in stopping people from spamming the chat room. It isn’t very complex, but it will give you the idea of what rate limiting can do. The algorithm for rate limiting is very simple, if a user sends too many messages in a certain interval, stop them from sending. At the top of chat_room.erl I’ve added the following:

-define(RATE_MSG_INTERVAL, 3).
-define(RATE_MSG_COUNT, 10).

I’ve just randomly chosen these values for now, and it means if you try to send more than 10 messages in 3 seconds, you will be rate limited. The check for rate limiting will go in the chat_message function:

handle_cast({chat_message, {Sess, Msg}}, State) when is_list(Msg) and(length(Msg) > 0) ->
    case get_session(Sess, State) of
        {error, not_found} -> {noreply, State};
        {ok, #client_state{nick=Nick,id=ID} = C} ->
            CleanMsg = chat_util:unicode_clean(lists:sublist(Msg, 256)),
            {NewC, NewState} = update_client(C, State),
            case handle_command(NewC, CleanMsg, NewState) of
                {ok, S1} -> {noreply, S1};
                _ ->
                    case should_rate_limit(NewC, NewState) of
                        {yes, {_, S2}} ->
                            send_system_msg(NewC, "You are sending messages to fast. Please wait a few seconds and try again."),
                            {noreply, S2};
                        {no, {RateClient, S2}} ->
                            chat_postoffice:broadcast_mail({msg, {chat_msg, {Nick, CleanMsg}}}, [ID]),
                            chat_postoffice:send_mail(ID, {msg, {sent_chat_msg, {Nick, CleanMsg}}}),
                            {_, S3} = update_client(RateClient#client_state{last_msg=now()}, S2),
                            {noreply, S3}
                    end
            end
    end;

We’ve just added a check that calls should_rate_limit and based on what it returns, the user is rate limited(and we notify them), or they are not, and the message can be sent. Also, you may notice the handle_command function, we will discuss this in a little.

To implement rate limiting, I’ve updated our client’s state with last_msg and msg_count:

-record(client_state, {
id, nick, host,last_action,admin=false,last_msg=never,msg_count=0
}).

Finally, the implementation of should_rate_limit. This function implements our simple algorithm.

should_rate_limit(#client_state{last_msg=never} = C, State) -> 
    NewState = update_client(C#client_state{last_msg=now(),msg_count=1}, State),
    {no, NewState};
should_rate_limit(#client_state{last_msg=LastMsg,msg_count=Count} = C, State) ->
	RateSecs = calendar:datetime_to_gregorian_seconds(calendar:now_to_datetime(now())) - calendar:datetime_to_gregorian_seconds(calendar:now_to_datetime(LastMsg)),
	case {RateSecs =< ?RATE_MSG_INTERVAL, (Count + 1) > ?RATE_MSG_COUNT} of
	    {true, true} -> {yes, update_client(C#client_state{msg_count=Count}, State)};
	    {false, _} -> {no, update_client(C#client_state{msg_count=0}, State)};
	    _ -> {no, update_client(C#client_state{msg_count=Count+1}, State)}
	end;
should_rate_limit(_, State) -> {no, State}.

Admin Control

This feature allows the administrator exercise some control over the chat server, with features such as kicking/banning users. The admin features are only protected by a password. We define the admin password as so:

-define(ADMIN_PASSWORD, "demopass").

It is probably a good idea to change this password if you are running it on a public server.

I have updated the client state to contain another field, called admin. When a user joins, this value defaults to false. To become an admin, (and to change a client’s state to admin=true), the user must authenticate. To handle authentication, and various other commands, we will use an IRC style /command param1 param2 param3..paramN. If you want to add spaces for a parameter, surround it with quotes.

You probably noticed in the previous section that we added a handle_command check in the chat message function. This will allow us to parse and handle any possible commands. It looks like the following:

handle_command(Client, Msg, State) ->
    case re:run(Msg, "^/([A-Za-z0-9]+) ?(.+){0,1}", [{capture, all_but_first, list}]) of
        {match, [Cmd]} -> process_command(Cmd, [], Client, State);
        {match, [Cmd, ParamStr]} -> process_command(Cmd, get_params(ParamStr), Client, State);
        _ -> {error, no_command}
    end.

It takes a chat message, and tries to parse out a command. If it is valid, we then process the command further. process_command takes the command (value after the slash), and a list of parameters (along with the client state, and server state). Using Erlang’s pattern matching, we can easily handle any command we want by just matching the string. For now, process_command handles two commands, /help, and /auth.

process_command("help", _, Client, State) ->
    send_system_msg(Client, "Commands: (/auth password)"),
    {ok, State};
 
% Generic admin handler.
process_command("admin", Args, #client_state{admin=true} = Client, State) -> process_admin_command(Args, Client, State);
 
% Authenticate an administrator    
process_command("auth", [?ADMIN_PASSWORD], Client, State) -> 
    send_system_msg(Client, "You are now authenticated as an administrator. Type /admin help for commands."),
    case Client#client_state.admin of
        false -> chat_postoffice:broadcast_mail({msg, {admin_logged_in, Client#client_state.nick}}, []);
        _ -> ok
    end,
    {_, NewState} = update_client(Client#client_state{admin=true}, State),
    {ok, NewState}; 
process_command("auth", _, Client, State) -> send_system_msg(Client, "Invalid adminstrator password."), {ok, State};  
 
% Unknown command. 
process_command(_, _, Client, State) -> 
    send_system_msg(Client, "Invalid command or format."),
    {ok, State}.

/help just prints out help. All users can access this command. The auth command is what will give us administrator access. Using pattern matching, we can check to see if the correct password was entered. If it was, we update the client state and notify all the other users that this user has became an admin.

Once a user is authenticated, the Generic Admin Handler function will match, and the process_admin_command function will be called.

Admin Commands

Now that the user is logged in, we want to add some admin functionality. In this tutorial, we add the following commands:

/admin logout
/admin help
/admin kick user reason
/admin ban user reason seconds
/admin unban host
/admin info user
/admin bans

Kicking a user

To kick a user, the admin will specify a user, and a reason. The code is:

% Kick a user
process_admin_command(["kick", User, Reason], #client_state{id=MySess} = Client, State) ->
    case get_user_by_nick(User, State) of
        {error, not_found} -> send_system_msg(Client, "Unable to kick: user not found.");
        {ok, #client_state{id=Sess} = C} when Sess /= MySess -> 
            send_system_msg(C, "You have been kicked from the chat room. Reason: " ++ Reason),
            chat_room:leave(Sess, "kicked: " ++ Reason);
        _ -> send_system_msg(Client, "Sorry, you can't kick yourself!")
    end,
    {ok, State};

First, we lookup the user. If the user exists, we *make* them leave the chat room, and specify the reason. That’s it.

Banning a user

This is a little more complicated, as we want to persist the ban between server restarts. To do this, we use a DETS table, which is basically ETS(Erlang’s in memory database), persisted to disk. Erlang stores the database in a file. We use the file name “bans.dets”. The table is created/opened in our chat_room init function:

init(_Args) -> 
	process_flag(trap_exit, true),
	{ok, BansTable} = dets:open_file("bans.dets", []),
	_ConnTable = ets:new(conns, [named_table]),
	timer:apply_after(?CHECK_IDLE_TIME, ?MODULE, find_idle_clients, []),
	{ok, #state{bans_table=BansTable}}.

Each item in the bans_table will be defined by the following record:

-record(user_ban, {
host,last_nick,reason,until
}).

Okay, now for the fun part. First, let’s see the function to ban a user:

% Ban a user
process_admin_command(["ban", User, Reason, Seconds], #client_state{id=MySess} = Client, State) ->
    case {get_user_by_nick(User, State), string:to_integer(Seconds)} of
        {_, {error, _}} -> send_system_msg(Client, "Unable to ban: invalid time specified.");
        {{error, not_found}, _} -> send_system_msg(Client, "Unable to ban: user not found.");
        {{ok, #client_state{id=Sess,host=Host} = C}, {Secs, _}} when Sess /= MySess -> 
            {Mega, S, Micro} = now(),
            BanTime = {Mega, S + Secs, Micro},
            BanTimeStr = httpd_util:rfc1123_date(calendar:now_to_local_time(BanTime)),
            send_system_msg(C, "You have been banned from the chat room until " ++ BanTimeStr ++ ". Reason: " ++ Reason),
            chat_room:leave(Sess, "banned: " ++ Reason),
            dets:insert(State#state.bans_table, #user_ban{host=Host,last_nick=User,reason=Reason,until=BanTime});
        _ -> send_system_msg(Client, "Sorry, you can't ban yourself!")
    end,
    {ok, State};

This function looks up the user, and if the user exists, we let them know that they have been banned(and disconnect them) and store an item in the DETS table describing the ban.

Now that the user has been disconnected, we need to make sure they can’t login until the ban is up. This check is implemented in our join function.

handle_call({join, {Nick, Host}}, _From, #state{clients=Clients} = State) when is_list(Nick) ->
    case {is_banned(Host,State), can_connect(Host), validate_nick(Nick, State)} of
        {{yes, Until}, _, _} -> {reply, {error, {banned, Until}}, State};
		{_, no, _} -> {reply, {error, too_many_conns}, State};
		{_, _, {error, Reason}} -> {reply, {error, Reason}, State};
        {_, {yes, NumConns}, {ok, ValidNick}} ->
            Session = get_unique_session(State),
            case chat_postoffice:create_mailbox(Session) of
                ok -> 
					update_host_conns(Host, NumConns + 1),
                    chat_postoffice:broadcast_mail({msg, {user_joined_room, ValidNick}}, [Session]),
                    Client = #client_state{id=Session,nick=ValidNick,host=Host,last_action=now()},
                    {reply, {ok, Session}, State#state{clients=[Client | Clients]}};
                {error, _} -> {reply, {error, not_available}, State}
            end
    end;

We have updated this function to call the function, is_banned, which determines if a user is indeed banned. If they are, we return an error, and the message is displayed to the user.

The ban function simply looks up a host/ip address in the bans table, and determines if it is still active.

is_banned(Host, #state{bans_table=Table}) ->
    Now = now(),
    case qlc:e(qlc:q([Ban || Ban <- dets:table(Table), Ban#user_ban.host == Host])) of
        [] -> no;
        [#user_ban{until=Until} | _] when Until >= Now -> {yes, Until};
        _ -> no
    end.

Connection Limiting

Finally, we’ve added a simple check in the join function, called can_connect, which checks to see if a user has the maximum connections allowed open. This is defined as:

% Max connections per host
-define(MAX_CONNECTIONS, 3).

The number of connections for a host is stored in an ETS table. We don’t need to persist this because if the server crashes, all connections will be gone anyway.

When a user joins, we update the connections table.

% Updates a host in connections ETS table
update_host_conns(Host, Conns) -> ets:insert(conns, {Host, Conns}).

When a user leaves, we update the connections table.

host_disconnected(Host) ->
	case ets:lookup(conns, Host) of
		[] -> ok;
		[{_, Num}] when is_integer(Num) -> ets:insert(conns, {Host, Num - 1});
		_ -> ets:insert(conns, {Host, 0})
	end.

Conclusion

Now we have a more functional server, but it still needs lots of work(that’s for you to decide). The source is available here. Try it out locally, by doing:

wget http://tech9computers.com/erl_chat-0.2.tar.gz
tar -xvzf erl_chat-0.2.tar.gz
cd erl_chat-0.2
make erl_chat

Then visit http://localhost:8000 to try it out

If anyone has any comments/suggestions/requests please let me know, I’d be happy to get some feedback on this. And also, you can try a demo here, although don’t think that demopass will get you admin control .

Building an Erlang chat server with Comet – Part 2

Chris Moos — Wed, 30 Sep 2009 01:01:34 +0000

You can see a live demo of the following tutorial at chat.tech9computers.com

Getting into the code

In the previous part, I introduced the chat system and how it should work. Now we are going to write the chat_postoffice, chat_mailbox, chat_room, and chat_web modules. Before you read download the source for this tutorial here. and dedicate about 20 minutes to read through this tutorial, because it’s kind of long (sorry).

Setting up the project

When you look at the source, take a look at the structure of the files and folders. There is a makefile to help you build and run the project. To run the server, just do the following:

$ cd erl_chat
$ make erl_chat

This will compile all the code, and start the OTP application. The web server runs on port 8000.

The Mailbox Process

Before we set up the post office, let’s define how a mailbox will work. First, it will run in its own process and keep a list of messages. Essentially it is just a loop, that accepts the following messages:

{mail, {msg, Data}}
{mail, {add_listener, {MsgID, Pid}}}
{mail, {remove_listener, Pid}}
{mail, {get_msg_id, Pid}}
quit

To create a mailbox, the post office will spawn a process and then the chat_mailbox:start function is called. The start function sets the initial state and then puts the process into hibernation.

start

start(ID) ->
    proc_lib:hibernate(?MODULE, loop, [#state{id=ID}]).

The following are the record definitions for the mailbox:

-record(message, {
id,data
}).
 
-record(state, {
id,cur_id=0,listeners=[],messages=[]
}).

NOTE: What is proc_lib:hibernate

This function helps keep the memory footprint much lower for a process. It is very useful when a process only does action when it receives a message. It discards the call stack and garbage collects for the process.

Now let’s write the loop function for our mailbox. It just receives a message for our process and does something with it. Here is the entire loop function.

loop

loop(#state{id=ID,cur_id=CurID,messages=Msgs,listeners=Listeners} = State) ->
    receive
        {mail, {add_listener, {_MsgID, _Pid} = Listener}} ->
            NewState = notify_listeners(State#state{listeners=[Listener | Listeners]}),
            proc_lib:hibernate(?MODULE, loop, [NewState]);
        {mail, {remove_listener, Pid}} ->
            NewListeners = lists:filter(fun({_Id, P}) -> P /= Pid end, Listeners),
            proc_lib:hibernate(?MODULE, loop, [State#state{listeners=NewListeners}]);
        {mail, {get_msg_id, Pid}} ->
            Pid ! {cur_msg_id, CurID},
            proc_lib:hibernate(?MODULE, loop, [State]);
        {mail, {msg, Data}} ->
            Msg = #message{id=CurID,data=Data},
            NewState = notify_listeners(State#state{messages=[Msg | Msgs],cur_id=CurID+1}),
            proc_lib:hibernate(?MODULE, loop, [NewState]);
        quit -> ok;
        _ -> proc_lib:hibernate(?MODULE, loop, [State])
    end.

Okay, let’s break down everything in that loop.

add_listener

When a new listener is added, the first thing we do is call notify_listeners, which will try to update the newly added listener(and any others) with any messages that have a higher/or equal message ID. Well go over notify listeners in a little.

remove_listener

This just goes through the list of listeners, if one matches the given Pid, remove it.

get_msg_id

This just returns the highest message ID (so the client knows what the latest message ID is).

msg

This adds a message to the mailbox. notify_listeners is also called to try and update anyone interested.

quit

Tells the mailbox to quit (and the process will exit).

Notifying listeners

Finally, our mailbox needs a notify_listener function. This will go through each listener and then check to see if any messages are available, and if so, send them to the Pid. If a listener is notified, it is removed also.

notify_listeners

notify_listeners(#state{listeners=Listeners,messages=Msgs,cur_id=CurID} = State) ->
    NewListeners = lists:filter(fun({MsgID, Pid}) ->
        case MsgID >= CurID of
            true -> true;
            _ ->
                % Select messages that are greater than or equal to the requested ID
                case lists:filter(fun(#message{id=ID}) -> ID >= MsgID end, Msgs) of
                    [] -> true; % no messages were found for this listener, keep it in the list
                    M -> Pid ! lists:map(fun(#message{data=Data,id=MID}) -> {MID, Data} end, M), false % remove it
                end
        end
    end, Listeners),
    State#state{listeners=NewListeners}.

Okay, that does it for our mailbox, now let’s work on the post office, which will communicate with our mailbox.

Creating the Post Office

Now it’s time to implement the post office. Without really focusing on the chat room aspects now, the post office should work somewhat like it does in real life. Ours will do the following:

Create/Delete mailboxes.
Send message to a mailbox by a unique ID (aka address).
Send message to all mailboxes (junk mail, broadcast).

Our chat_postoffice module uses the gen_server behavior so a lot of the code in the module is related to that. First let’s define the state of our postoffice server. It is very simple and just contains a list of mailboxes.

-record(state, {
mailboxes=[]
}).

Each item in the mailboxes list will be a tuple:

{Id,Pid}

Where Id is the mailbox ID, and Pid is the process ID for the mailbox. You can see in our init function that our server starts with no mailboxes(empty list).

Our create_mailbox function is defined below. First we check to see if the mailbox already exists, if not, we add it to our list and spawn off a mailbox process (we will go into the chat_mailbox module in a little bit).

create_mailbox

handle_call({create_mailbox, ID}, _From, #state{mailboxes=MBoxes} = State) ->
    case get_mailbox(ID, State) of
        {ok, _} -> {reply, {error, already_exists}, State};
        {error, notfound} ->
            Pid = spawn_link(chat_mailbox, start, [ID]),
            NewBox = {ID, Pid},
            {reply, ok, State#state{mailboxes=[NewBox | MBoxes]}}
    end;

And here is our method for deleting a mailbox. It removes the mailbox from the list and sends a message to the mailbox process to quit.

delete_mailbox

handle_cast({delete_mailbox, ID}, #state{mailboxes=MBoxes} = State) ->
    NewBoxes = lists:filter(fun({Id, Pid}) ->
        case Id /= ID of
            false -> 
                % tell the mailbox process to quit
                Pid ! quit, false;
            _ -> true
        end
    end, MBoxes),
    {noreply, State#state{mailboxes=NewBoxes}};

Now we want to create methods for sending and broadcasting mail.

To send mail, we look up the mailbox ID, and send the process a message. Note: Delivery is not guaranteed, if the mailbox process dies then it will never receive the message. In another post we will make it more fault tolerant, but for now, I wouldn’t worry about the mailbox process dying.

send_mail

handle_cast({send_mail, {ID, Msg}}, State) ->
    case get_mailbox(ID, State) of
        {ok, {_Id, Pid}} -> Pid ! {mail, Msg};
        _ -> ok
    end,
    {noreply, State};

And finally, let’s make our broadcast function. It takes a message and a list of ID’s not to send to.

broadcast_mail

handle_cast({broadcast_mail, {Msg, Except}}, #state{mailboxes=MBoxes} = State) when is_list(Except) ->
    [Pid ! {mail, Msg} || {Id, Pid} <- MBoxes, lists:member(Id, Except) == false],
    {noreply, State};

chat_room server

This server contains the core logic of our chat room. This includes maintaining all connected users, and functions to allow a user to join or leave the chat room, get list of online users, and send/receive messages(and events).

The state of the chat_room server is defined below. At the most basic level, it is just going to contain a list of all the users in the chat room. The client state is also defined below.

-record(state, {
clients=[]
}).
 
-record(client_state, {
id, nick, host,last_action
}).

The chat_room server will have the following functions:

join
leave
wait/wait_finish
chat_message
get_users
get_msg_id
find_idle_clients

Let’s start by writing the join function. This will first check to make sure that a nickname is valid(alphanumeric, no longer than 16 characters) and available. If everything is okay, we will create a mailbox for the user, notify all other users that a user has joined, and return a unique session to the caller. If an error occurs, it is returned to the caller as well.

join

handle_call({join, {Nick, Host}}, _From, #state{clients=Clients} = State) when is_list(Nick) ->
    case validate_nick(Nick, State) of
        {error, Reason} -> {reply, {error, Reason}, State};
        {ok, ValidNick} ->
            Session = get_unique_session(State),
            case chat_postoffice:create_mailbox(Session) of
                ok -> 
                    chat_postoffice:broadcast_mail({msg, {user_joined_room, ValidNick}}, [Session]),
                    Client = #client_state{id=Session,nick=ValidNick,host=Host,last_action=now()},
                    {reply, {ok, Session}, State#state{clients=[Client | Clients]}};
                {error, _} -> {reply, {error, not_available}, State}
            end
    end;

The leave function is used to log out of the chat room. It will remove the user, and notify the others that that user has left. In the next part, we will implement a timeout so that if a user hasn’t done anything in a specified time, he will be removed.

leave

handle_cast({leave, {Sess, Reason}}, #state{clients=Clients} = State) when is_list(Reason) ->
    case get_session(Sess, State) of
        {error, not_found} -> {noreply, State};
        {ok, Client} ->
            chat_postoffice:delete_mailbox(Client#client_state.id),
            CleanReason =  chat_util:unicode_clean(lists:sublist(Reason, 32)),
            chat_postoffice:broadcast_mail({msg, {user_left_room, {Client#client_state.nick, CleanReason}}}, [Client#client_state.id]),
            OtherClients = lists:filter(fun(#client_state{id=ID}) -> ID /= Client#client_state.id end, Clients),
            {noreply, State#state{clients=OtherClients}}
    end;

Chat message will broadcast a message to the chat room.

chat_message

handle_cast({chat_message, {Sess, Msg}}, State) when is_list(Msg) ->
    case get_session(Sess, State) of
        {error, not_found} -> {noreply, State};
        {ok, #client_state{nick=Nick,id=ID} = C} ->
            CleanMsg = chat_util:unicode_clean(lists:sublist(Msg, 256)),
            chat_postoffice:broadcast_mail({msg, {chat_msg, {Nick, CleanMsg}}}, [ID]),
            chat_postoffice:send_mail(ID, {msg, {sent_chat_msg, {Nick, CleanMsg}}}),
            NewState = update_client(C, State),
            {noreply, NewState}
    end;

Finally, we have ,wait, wait_finish, and get_users.

The wait function will add a listener to the user’s mailbox.

wait

handle_cast({wait, {Sess, MsgID, Pid}}, State) when is_integer(MsgID) ->
    case get_session(Sess, State) of
        {error, not_found} -> Pid ! {error, bad_session}, {noreply, State};
        {ok, C} ->
            NewState = update_client(C, State), 
            chat_postoffice:send_mail(Sess, {add_listener, {MsgID, Pid}}), {noreply, NewState}
    end;

wait_finish removes the listener from a mailbox.

wait_finish

handle_cast({wait_finish, {Sess, Pid}}, State) ->
    case get_session(Sess, State) of
        {error, not_found} -> {noreply, State};
        {ok, _} -> chat_postoffice:send_mail(Sess, {remove_listener, Pid}), {noreply, State}
    end;

The get_users function just returns a list with the nickname of every user.

get_users

handle_call({get_users, Sess}, _From, State) ->
    case get_session(Sess, State) of
        {error, not_found} -> {reply, {error, not_found}, State};
        {ok, C} -> 
            NewState = update_client(C, State),
            {reply, {ok, lists:map(fun(#client_state{nick=Nick}) -> Nick end, State#state.clients)}, NewState}
    end;

That does it for the chat_room server. If you look through the code you will also see a find_idle_clients function, which runs on a timer to see if any clients have left the browser and timed out.

Finally, the web front-end

There are two pages that the end user will see. The login page, where they can choose a nickname, and the chat page. There are two templates that we will be using, called index.html, the login page, and chat.html, the chat screen. The rest of the interaction between the web browser and the server will take place with AJAX.

First, let’s make it so a user can join the chat room. Take a look at the chat_web module and you will see a section called Request Handlers. These are functions that are called when a request comes in. We can look at the path to decide what to do.

At the bottom of our request handlers, we have one that will just try to serve a file from our docroot directory. This is where we will store our javascript and any other static files.

handle_request(Req, Path) ->
	Req:serve_file(string:sub_string(Path, 2), "docroot", []).

Also, I’ve included some helper functions in chat_util for loading templates with ErlyDTL. You can use the Django template language in your templates, and then render them with any variables you want. We don’t use this that much.

Login

Our login page, which is at the path /, is rendered like so:

handle_request(Req, "/") -> html_ok(Req, chat_util:get_template("index", []));

You can take a look at the template, it basically does a simple HTTP post to the path, /login/.

Our login handler just calls chat_room:join to attempt to join the chat room. If an error occurs, it is passed into the login template and returned to the user. If all is good, we set a cookie with the user’s chat session ID.

login

handle_request(Req, "/login/") ->
    Post = Req:parse_post(),
	case chat_room:join(chat_util:get_parameter("nick", Post), Req:get(peer)) of
	    {ok, SessID} -> 
	        SessCookie = mochiweb_cookies:cookie("chat_sess", SessID, [{path, "/"}]),
	        Req:respond({302, [SessCookie, {"Location", "/chat/"}], <<>>});
		{error, not_available} -> html_ok(Req, chat_util:get_template("index", [{error, "The nickname is not available."}]));
	    _ -> html_ok(Req, chat_util:get_template("index", [{error, "The nickname must be alphanumeric and not blank."}]))
	end;

The main chat interface

Once the user is redirected to the /chat/ path and has a session cookie set, the AJAX will take over. I don’t want to go too much into the javascript, but it mainly does the following:

Get’s the current message ID and validates the session.
Get’s the list of online users.
Waits for any messages to come in from the server.

Start client is called to get the message ID. If successful, it gets the online users.

startClient

function startClient() {
	addSystemMsg("Establishing connection to chat service...");
	new Ajax.Request('/chat/start/',
	  {
	    method:'get',
	    onSuccess: function(transport){
			var response = transport.responseText.evalJSON();
 
			if(response.status == "ok") {
				msgID = response.response;
				addSystemMsg("Connected to chat service.");
				getOnlineUsers();
			}
			else if(response.status == "error" && response.response == "bad_session") addSystemMsg("Unable to locate your session. Please \"/\">login again.");
	    },
	    onFailure: function(){ addSystemMsg("The chat service is not responding at this time.") }
	  });
}

After the message ID and the online users are retrieved, the client enters the message loop. After a message is received, it is processed by handleServiceMsg, which basically looks at the JSON object and decides what to do(user joined, chat message, etc,.).

getServiceMsg

function getServiceMsg() {
	new Ajax.Request('/chat/wait/?msg_id=' + msgID,
	  {
	    method:'get',
	    onSuccess: function(transport){
	        if(transport.responseText == "") {
	            setTimeout('getServiceMsg();', 10000);
	        }
	        else {
	            addSystemMsg(transport.responseText);
	            var response = transport.responseText.evalJSON();
	            if(handleServiceMsg(response) == true) {
		  	        getServiceMsg();
		        }
	        }
	    },
	    onFailure: function(){ 
	        setTimeout('getServiceMsg();', 10000) }
	  });
}

getServiceMsg talks to the following handler, which just adds a listener for our session, and waits for a message.

handle_request(Req, "/chat/wait/") ->
    MsgID = chat_util:get_parameter_int("msg_id", Req:parse_qs()),
    chat_room:wait(get_session(Req), MsgID, self()),
    timer:apply_after(?COMET_TIMEOUT, ?MODULE, timeout_wait, [self()]),
    proc_lib:hibernate(?MODULE, wait, [Req]);

If there are no messages after a certain time(?COMET_TIMEOUT), a timeout message is sent and the client is told to reconnect.

When the user wants to send a message, an AJAX post is performed to the path /chat/send_msg/. The following is the handler for sending a chat message. It’s pretty simple.

handle_request(Req, "/chat/send_msg/") ->
    chat_room:chat_message(get_session(Req), chat_util:get_parameter("msg", Req:parse_post())),
    json_respond(json_client_ok(<<"">>), Req);

Conclusion

Well, that’s it for now. You can download the source and try it out yourself. The server runs on port 8000. (http://localhost:8000).

In the next part, I will discuss some more advanced features, such as rate limiting, limiting connections for a host, admin control, more fault tolerance(using ETS/DETS) for storing messages, and more. While you are waiting, come chat: chat.tech9computers.com