Campfire and Jabber

Note: this post is a work in progress.

Last night at RUM there was a presentation (by unknown) which addressed the issue of “telecomputing” (trendy word for telecommuting), using Rails + Trac + Campfire on a project with a global development effort as a model. There were a few major points to his project's successful approach:

Use Conventions From Rails Manual
”Convention over configuration” is the motto Rails was developed by. The presenter addressed an all-too-familiar sentiment of seeing Ruby degenerate into something resembling PHP, as soon as the code deviates from the standard conventions too much. To nip this issue in the bud, his team adopted a strictly convention-centric approach--if it's not in the manual, don't do it. (note: I don’t actually know what book he’s referring to, please enlighten me if you know)

Use a "Torch Passing" Technique for Global Development Teams
This was the first time I had heard the term "torch passing" in this context, but I think it is a perfect analogy for its purpose. Torch passing is what it sounds like: having people in one location brief the next team in chronological order about the day's work, and hand leadership to the new them. In this way, with no central team leadership, a global development effort is not working headless for 24-48 hours at a time.

One member of the audience shared his belief that communication will never be as good as when a person is working next to you, which I think can be true depending on the task at hand. His specific example was when a designer made incorrect attempts to fix defects in a GUI; that had 48 hours wasted between each iteration. Perhaps if leadership were passed around the globe throughout the day, rather than remaining static, this issue would disappear.

Use Skype for conference calls
Skype is cheaper than phone service (free, in fact), and multi-platform. And Dan Grigsby made an interesting contribution to the subject: a study found that effectiveness of communication increases with fidelity (citation?), and Skype has about twice the fidelity of a standard phone line.

Campire
This where the meat of my interest lay. Campfire is a 37signals application that is very much like an IRC client. It is more like a Jabber MUC, however, in that conversations remain static and history is sent to new clients that enter a chat room (the past X lines, minutes, etc). After briefly looking at the video tour available on 37signals' website it seems like there are some interesting nuggets that can be added to Soashable. Namely file uploading that displays rich media directly in the chat window intelligently, presumably by mime type. Indeed, most of the product could be implemented using existing XEP-defined functionality.

I think there is lots of potential here to build a clone on the same messaging/presence platform as Soashable using existing Jabber extensions which, as I love, means minimal work on my part . As far as business goes, I have some resevations about an organization like DHS and some of the employees' (business analysts, management, etc), willingness to adopt something remotely similar to Campfire (I could be wrong). But for a smaller team in a more cutting edge environment, I think it would be great and very welcome. I can't count the number of walks across the office it would have saved me at my former employer. Maybe if there was an ugly, unintuitive interface, like Oracle products, a place like DHS would like it more ;)

I think the core advantages of using Jabber over a custom Java/Rails backend is scalability, mitigation of labor and loose coupling. By using an existing, standardized, specified platform you don't have to do a lot of things yourself. And if you do, you already have templates to explain your custom functionality.

Using Meebo as an example: they have a team of C++ developers maintaing a custom version of GAIM on their servers, and a nasty (to maintain) looking Ajax/JSON protocol. Okay, but they need scalability. With Jabber, smarter people already have that taken care of. What about the mobile interface they promised forever and a day ago? Part of me thinks that they coupled their protocol too tightly with their GUI. I was able to set up a working, web-based AIM client over the course of a weekend with little prior knowledge of Jabber. I was then able to unit test it and completely rebuild the GUI without changing a line of tested code.

You'd tell me if I were stuck in a box where all I could see is Jabber, right?

Update: Javascript XMPP and Data Plans

Just a quick update:

Over the past weeks I have been working on a new Jabber client written in Javascript. It will be a successor to JSJaC in the Soashable project. The API is designed with Smack in mind, but deviates in that it uses DOM ubiquitously rather than building XML strings. However it is still designed such that one could implement a packet and simply override getNode() to create a new XML node on the fly.

It is mostly unit tested, and lack of testability was a core reason for finally leaving JSJaC. The approach is to create a DummyXmppConnection, a subclass of XmppConnection that doesn't actually make Ajax requests, and test output queue (expected vs actual) and inject responses to test for events.

I've also added a data plan to my cell phone service. My device is a T-Mobile MDA connecting via EDGE, and I get speeds of 100±30 kbps. From limited test runs, it is a bit slower than the iPhone. I installed Opera Mobile and it seems to handle Ajax pretty welll; most tests for XMPP client pass, where they don't even run for IE mobile.

I've got a lot of new ideas, and you can see where this is going :) I have more for you later.

How unit tests saved my night

Last night I started working on a new XMPP library for Javascript; I'll explain later ;). Anyway, I was wrapping it up for the night, separating Javascript from the monolithic HTML file I was working in (because I'm Agile like that), when I accidentally deleted the bottom half of my code and saved without realizing (doh!).

Tragic story, but the funny part is that I had unit tests for all the code I deleted, so within about 15 minutes I was able to re-build all lost functionality. My tests weren't fantastic, but they at least covered basic expectations of the methods' output.

Unit tests FTW.

XEP-0049 - Private XML Storage

I just implemented XEP-0049 - Private XML Storage for my Jabber client (source here). What it does is allow a client to store arbitrary XML data in a given namespace and retrieve it using the same.

To demonstrate a creative usage of it, I added an 'onconnect' event that will run any Javascript in 'soashable:userscript:event:signon'. Maybe stupid or dangerous; I haven't considered the implications carefully; but it's just a proof of concept.

First Official PostgreSQL Benchmark Released

I'm happy to hear this news and glad they waited until they were ready for their numbers to be official. Article: here.

This publication shows that a properly tuned PostgreSQL is not only as fast or faster than MySQL, but almost as fast as Oracle (since the hardware platforms are different, it's hard to compare directly). This is something we've been saying for the last 2 years, and now we can prove it.

...

The difference in cost, per pricing given by Tom Daly, is much more substantial than the difference in performance. Our benchmark run costs about $65,500 for the hardware, and has no software acquisition cost. Based on retail price listings, however, the Oracle/HP run costs $74,000 for the hardware and $110,000 for the software. That's a almost 200% more expensive. Add annual maintenance and support and it becomes even greater.

So effectively, you can still get slightly better performance by using Oracle on commodity hardware. You just pay $120,000 to gain an incremental amount of performance. But then, you already knew that, didn't you?

Soashable Source Code Released

I packaged up the source code to Soashable, a Meebo-like Jabber client that I've been tinkering with (previous post). It's available on Google Code, here. Note that I'm not proud of the quality of the code, but it works ;)

A working copy of the application is online here (could be down/broken at any time).

At the time of this writing, it supports:

• Registration
• Sign-on
• Send/Receive IM
• Add Contact
• Roster with groups and presence updates (read-only)
• Service Discovery (XEP-0030)
• Gatway Discovery (XEP-0100, no registration yet)
  

Here is the new Roster which was no in the previous screen shot:

Enjoy!

Untapped Instant Messaging Features

As I've read various Jabber books and specs over the past few days I have come across a few gems that have glistened "next ubiquitous feature," in so many words.

Offline messages - Yahoo! and ICQ have done this for quite a while, but it never seemed to catch on with AOL. With this feature, a message is delivered to its recipient even if they are not online when the message is received. If they are offline, the message put into a message queue, tagged with an "originally sent" date element, and delivered when the recipient is available.

"Displayed" notices - "your buddy is typing a message" lurkers can breathe a sigh of releif, as they no longer have to anticipate whether the other party saw their message. When the displayed element is included with a message, the remote client will send a response back when it believes the other party has displayed the message. This would likely be when the other person clicks/brings the message into focus. Being a component of message, these are compatible with offline messages in both directions.

Message expiration - Jabber lets a message expire after a set amount of time. If the client were to implement the expiration policy, it would destroy a message at the correct time. Likewise, servers support this feature with offline messages and do not deliver expired messages.

Suspend / Resume - This is unique to HTTP bound transport layers (XEP-0124), but is a nice feature that Meebo doesn't currently support. In short, it lets a user leave a site and resume their session upon return (obviously client UI state will be lost). From my understanding, the server queues messages on the remote end as it would with an Offline message and delivers them upon a Resume command. This behaviour would likely be handled by the plug-in rather than the server itself, since they appear "available" between suspend and resume. There is no upper bound as to how long a connection can be suspended.

While none of these features are extraordinarily killer, I think that once discovered they will become ubiquitous.

The Mindset of an Idea

Right now I'm at one of those interesting times where I am creating a piece of software without knowing what I'm making until it's done. The code base is haphazardly organized at best--originally starting off as a single HTML file with JS objects and then branching out into a collection of JS files roughly split up by class (helpers go with their main module even if shared by multiple, etc). This time produces some of the worst code I ever write. There are shining examples of ingenuity strewn throughout, but no coherent strategy or consistency whatsoever.

Some variables are camel case, some are underscored. Some strings use single quotes, while others are double. Global variables are used. Mutators are not used. There is no glossary of terms (sometimes it's a buddy, sometimes it's a contact). It's the kind of stuff that could ruin a reputation if it fell into the wrong hands. But it works, and it's evolving. And if it's a good idea that's worth running with, it will be rewritten to be beautiful. If not, the small patches of beauty and perfection will be tossed into a junk heap of old projects that will never be salvaged or even salvageable if I were to try. After a couple weeks of not working on the project, it will be forever lost.

Now is the perfect time to capture "requirements" and design the architecture because an intimate knowledge of the new tools is fresh in my mind. But I'm too busy working hard and evolving the idea.

But there is a mindset associated with this level of dedication to a project, being 'in the zone' for 12+ hours a day. It makes me better at everything. It makes me more confident. It makes me seriously think about quitting my job with no business plan and no idea where I will get my next paycheck. It makes me write in a style I don't usually write in, for better or worse. I'm writing this article like I've been writing my code, full of good, solid ideas but absolutely lacking cohesion. Most importantly, it makes me behave with a drunken sense of enlightenment :)

Hardware Scare

The past few days have been loaded with heavy, intense development too time consuming to worry about piddly things like backup or version control. Just as I was preparing a sample of my app for a friend, my server shut down unexpectedly and wouldn't start back up. The most likely time for a failure to occur is when you are depending on the equipment the most. This is the third piece of hardware that has died in the past couple months. Perhaps I'm getting power spikes from the central air unit, or perhaps my server room (aka my bedroom closet) is getting too hot. Erring on the side of caution, I opted to replace the hardware and get a UPS for its power cleaning properties. I ended up going with an APC 650VA that should give me about an hour of battery time, send a signal via USB to my server, and $50k equipment and data protection, an added bonus.

The machine ended up turning back on mysteriously after about fifteen minutes, so perhaps it was overheating. But now at least I have peace of mind knowing that the AC isn't going to blow away my hardware or data. I've also got added protection against data loss during power failures, since I am now running apcupsd which will shut my system down in event of a power failure and depletion of battery power.

I was scared and frustrated by the vulnerability I felt when my machine died. Luckily I've already taken the proactive measure of using rsync to back up my data, so I am now as fault tolerant as ever.

Meebo Clone -- as Easy as it Seems?

Once again I've started kicking around the idea of creating a web based chat system. After seeing the possibilities from the folks at Meebo, I am starting to tinker with a prototype based on Jabber + HTTP Binding (XEP-0124). This standard allows Ajax requests to be made directly to the jabber server (or thought an HTTP proxy to the server), cutting out any effort required to make a custom communication layer. Further, it allows the full extensibility of Jabber to be realized, since raw packets are delivered directly to the web client. It also allows for sessions to be suspended and resumed, which means that if a person refreshes the browser window or leaves the site entirely and returns, they never have to sign off of the network because they retain a session key stored as a cookie. Their browser simply received messages that were queued up after their suspension upon a call to resume (disclaimer: I haven't fully tested suspend/resume functionality).

From my understanding (which comes from job postings and looking at JS code), Meebo uses a modified version of Gaim with a custom Ajax communication layer. I question why they didn't take the approach that I am taking, but think it could be to do with some factor I haven't taken into account. So far though, my experimentation has been very fruitful:


In the past 12 hours I have been able to piggy back off of existing tools to make a server that supports communication with external gateways (AIM, MSN, et al), and a client that is able to register, sign on, send and receive IMs, and receive roster updates. Not bad, if I do say so myself.

No estimate or time line about when source code will be released.