No PhD

I work for Nature Publishing, but I haven't got a PhD

Rails 3 is going to rock

with 2 comments

Yehuda Katz presented the keynote at Rails Underground this morning, and very interesting it was too. The core of his speech revolved around the new version of Rails. Added to the usual tenets of convention over configuration, DRY, etc was a new concept. Opt-in.

Rails 3 has been broken down from a monolithic framework to a set of loosely coupled modules tied together with stupidly simple public interfaces. The router is a module, the renderer is a module, validation is a module, etc, etc. Any can be used or not in your Rails app, or indeed your vanilla Ruby/JRuby/MacRuby app. This direction is beneficial for many reasons:

  • New Rails projects can just pull in the parts of the framework they need.
  • Existing projects can easily remove dependencies on parts of the framework they don’t need.
  • Rails components can be swapped out for alternative implementations

He showed examples of how Sinatra can now effectively be implemented in Rails, how ActiveSupport has been broken up to be more useful outside Rails, and how ActiveModel has replaced ActiveRecord. Very interesting, and very promising. Apparently the APIs will be settled within the next few weeks so edge Rails is going to be stabilising in terms of the interfaces.

One thing is for sure. Rails is going to be a a very different beast.

Written by spanx

25th July, 2009 at 1:03 pm

Posted in Code

Tagged with

Clojure Robot on Google Wave

with 8 comments

I’ve wanted a little project to start learning Clojure in a bit more detail, and having got access to Google Wave a couple of weeks back, I figured a suitable challenge would be to build and deploy a simple Wave robot to Google App Engine. So I did. And may I present… CENSOR BOT!

Censor Bot (censorbot@appspot.com if you have a Wave account) listens to messages posted in a wave and removes naughty swear words. It’s a pretty simple beast (about 40 lines of Clojure in all), but is non-trivial enough to serve as a good first project. In this post I’ll go through the process of getting a Clojure Wave robot built, hosted on GAE and accessible in your wave account.

Firstly we have some dependencies:

  • Java 6
  • Ant
  • A Google App Engine account (sign up here)
  • A Google Wave account
  • The Google App Engine SDK
  • Clojure 1.0

Firstly, download and install the Google App Engine SDK from here. Just unzip the SDK (latest is v1.2.2) somewhere useful. Whilst you’re there, it’d be a good idea to set up your GAE account too. Once you’re signed in (at http://appengine.google.com) you can click the “Create an application” button and reserve your app identifier. Note this down as you’ll need it later.

Next, ensure you’re running Java 6; you should see the following:

$ java -version
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03-211)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02-83, mixed mode)

If you’re running a pre Java 6/JDK 1.6 version, you’ll need to upgrade as the Wave API only runs on Java 6. I suffered a couple of gotchas here on OS X. Firstly, open Applications/Utilities/Java Preferences.app and update the list so Java 6 is selected. This ensures code runs under that version of the JVM. However, Ant will still try and compile under JDK 1.5. You have to adjust your JDK symlinks too.

$ cd /System/Library/Frameworks/JavaVM.framework/Versions
$ sudo rm CurrentJDK
$ sudo ln -s 1.6 CurrentJDK

You’ll also need to download and install Clojure. Instructions here.

Now the fun can begin. cd to your favourite project location and run:

$ git clone git://github.com/indmill/censorbot.git

This has everything you need to write a basic Clojure Wave robot. The only things you’ll need to change are:

  • In build.xml, update the sdk.dir property to point to your GAE SDK location.
  • In war/WEB-INF/appengine-web.xml, change the application to be your GAE application identifier.
  • That’s pretty much it, unless you’re feeling brave and want to rewrite my newbie Clojure code.

You should just be able to run:

$ ant deploy

now to deploy the app to GAE. It may complain that your GAE credentials are not stored. I had to run:

$ path/to/gae-sdk/bin/appcfg.sh update war

And voila, you should be able to add your newly deployed censorbot to a Wave and off you go. The censorbot doesn’t censor many actual rude words yet, but feel free to edit its dictionary.

The code itself is very straightforward if you have done any Clojure. The main entry point from Wave-land is the function:

(defn -processEvents
  [this bundle]
  (let [wavelet (.getWavelet bundle)]
    (welcome-if-new bundle wavelet)
    (doseq [event (blip-submitted-events (.getEvents bundle))]
      (censor (.getBlip event)))))

It retrieves the wavelet from the event bundle, welcomes itself to the wave then filters out BLIP_SUBMITTED events and censors the contents. I’ve tried to make it as functional as I can but proper Clojure hackers out there could probably tidy it up a lot. Anyway, I welcome any comments, and if my instructions have missed any steps let me know and I’ll improve. Thanks.

Written by spanx

23rd July, 2009 at 5:01 pm

Posted in Code

Early stage developers

leave a comment »

Early Stage Developers
As an ex-freelancer, and somebody who still receives the odd request I have grown weary of approaches of the following ilk: “We’ve got this great idea, we just need a developer to build it for us. We haven’t got any cash but we’re willing to offer a small equity share”. These approaches are designed make $ signs spin around in the developers eyes. However, my gut feeling when I receive an approach like this is:
1. If the business guy won’t risk any cash (whether it’s his or somebody elses) on the big idea then it’s probably a crappy idea.
2. All the business guy is trying to do is offload the risk of failure to me.
As a business guy, when raising cash for a new enterprise you would typically search for investors to put in money in return for a percentage of your company. Early stage investors usually put up relatively small amounts of cash for larger percentages of the company. This is because the perceived *risk* – at an early stage the company/idea is incredibly unlikely to succeed – makes it effectively worthless. When they approach a developer to build the product at this early stage and all that is offered is equity they are asking them to take the same risk, with something just as, if not more valuable than money. Time.
I can see why this is an attractive route for business guys. Why take investment to fund development of your product and lose a great chunk of your company when you can go straight to the developers and get the product built for a much smaller percentage? But the approach is flawed in many ways:
– Good developers always get work. Proper paid work. You’re not going to get the best developers.
– Given a fixed percentage equity, developers will cut corners to deliver features in the shortest time.
– If the relationship sours there’s a lack of flexibility on how to proceed without resorting to legal channels
Business guys trying to develop their products in this way need to realise one thing. The early stage developer is an investor and thus should be treated in a similar way; someone to pitch to, someone to get advice from, someone to be completely transparent with. Someone to trust. And thus someone to treat with respect.
Now I’ve been lucky with the jobs I’ve selected in the past. I hooked up with very passionate people who had (admittedly limited) funding, who had a great idea that I shared the passion for, and who have been very transparent in their dealings with me. Even given that however, there are certainly things I would do differently now.
Firstly, the business guy *pays* for the first iteration (which they did for the gig I accepted). No question. This way you have closure if the relationship doesn’t work out for whatever reason. You get paid up front, you do the work to contract, then you stop. How much is up to you, but I suggest a daily rate that works for you. At the end of this iteration the business guy should have a prototype or v1, and the developer has been paid.
If the first phase works out and they like what you’ve delivered, we have a basis for a relationship. Now it’s time to barter equity for features; DON’T accept a flat equity percentage for continual involvement. Less “Give me 20% and I’ll work for you”, more “I will build you the commenting system for 5% equity in your company”. I think this approach is great for many reasons:
– the business guy is focussed on getting a useful feature, and the developer is focussed on delivering it.
– the business guy gets more flexibility – if he gets funding in the future, then future iterations can be paid for in real money.
– the relationship has flexibility in the event it sours as the only agreement is to deliver *the next iteration*.
This iterative model can continue until the product is complete, the product fails, the developer gets hired as the CTO, whatever.
I think there’s a seed of a good business venture in there somewhere. Any developers out there interested? There’s a small equity share in it for you…

As an ex-freelancer, and somebody who still receives the odd request I have grown weary of approaches of the following ilk: “We’ve got this great idea, we just need a developer to build it for us. We haven’t got any cash but we’re willing to offer a small equity share”. These approaches are designed make $ signs spin around in the developers eyes. However, my gut feeling when I receive an approach like this is:

  1. If the business guy won’t risk any cash (whether it’s his or somebody elses) on the big idea then it’s probably a crappy idea.
  2. All the business guy is trying to do is offload the risk of failure to me.

As a business guy, when raising cash for a new enterprise you would typically search for investors to put in money in return for a percentage of your company. Early stage investors usually put up relatively small amounts of cash for larger percentages of the company. This is because the perceived risk – at an early stage the company/idea is incredibly unlikely to succeed – makes it effectively worthless. When they approach a developer to build the product at this early stage and all that is offered is equity they are asking them to take the same risk, with something just as, if not more valuable than money. Time.

I can see why this is an attractive route for business guys. Why take investment to fund development of your product and lose a great chunk of your company when you can go straight to the developers and get the product built for a much smaller percentage? But the approach is flawed in many ways:

  • Good developers always get work. Proper paid work. You’re not going to get the best developers.
  • Given a fixed percentage equity, developers will cut corners to deliver features in the shortest time.
  • If the relationship sours there’s a lack of flexibility on how to proceed without resorting to legal channels

Business guys trying to develop their products in this way need to realise one thing. The early stage developer is an investor and thus should be treated in a similar way; someone to pitch to, someone to get advice from, someone to be completely transparent with. Someone to trust. And thus someone to treat with respect.

Now I’ve been lucky with the jobs I’ve selected in the past. I hooked up with very passionate people who had (admittedly limited) funding, who had a great idea that I shared the passion for, and who have been very transparent in their dealings with me. Even given that however, there are certainly things I would do differently now.

Firstly, the business guy pays for the first iteration (which they did for the gig I accepted). No question. This way you have closure if the relationship doesn’t work out for whatever reason. You get paid up front, you do the work to contract, then you stop. How much is up to you, but I suggest a daily rate that works for you. At the end of this iteration the business guy should have a prototype or v1, and the developer has been paid.

If the first phase works out and they like what you’ve delivered, we have a basis for a relationship. Now it’s time to barter equity for features; DON’T accept a flat equity percentage for continual involvement. Less “Give me 20% and I’ll work for you”, more “I will build you the commenting system for 5% equity in your company”. I think this approach is great for many reasons:

  • the business guy is focussed on getting a useful feature, and the developer is focussed on delivering it.
  • the business guy gets more flexibility – if he gets funding in the future, then future iterations can be paid for in real money.
  • the relationship has flexibility in the event it sours as the only agreement is to deliver *the next iteration*.

This iterative model can continue until the product is complete, the product fails, the developer gets hired as the CTO, whatever.

I think there’s a seed of a good business venture in there somewhere. Any developers out there interested? There’s a small equity share in it for you…

Written by spanx

22nd July, 2009 at 8:31 pm

Posted in Code

Tagged with ,

HSBC Secure ePayments for ActiveMerchant

with 3 comments

Something else I’ve been doing recently is refactoring the credit card processing code in Indmill (http://indmill.com). As we are in the process of providing Paypal payments in addition to our HSBC payments system it made sense to finally migrate to ActiveMerchant. However, ActiveMerchant doesn’t support the HSBC Secure ePayments API… until now!

I’ve pretty much implemented everything I need for Indmill, so there are a couple of bits of the API that need doing, but it’s available at http://github.com/indmill/active_merchant for now. Once I feel it’s properly finished I’ll try and push it to the core project.

Fixes, improvements, etc would be welcomed and hugely appreciated.

Written by spanx

20th July, 2009 at 12:38 pm

Posted in Code

An introduction to Cucumber

leave a comment »

We’re using Cucumber internally at Nature Publishing to define requirements and run integration tests. It’s becoming embedded in the way we drive our development iterations. I wrote this originally as a Wiki page to help introduce the concepts and figured, hey, I might as well share it. So here it is…

Cucumber is a Ruby gem/framework that enables documents written in a structured plain text language to be executed as automated integration tests. As ever, an example is the best way to explain:

Feature: Authenticating users
  In order to manage my account, add events, etc
  As a Organizer
  I want to log in to the service

  Scenario: Log in with invalid login
    Given the following users exist: 
      | email | password | first_name | last_name | title | country |
      | i@nature.com | itsasecret | Bob | Smith | mr | UK |
      And I view the login screen
    When I fill in "email" with "withnail@nature.com"
      And I fill in "password" with "anothersecret"
      And I press "Log in"
    Then I should view the login screen
      And I should see "Could not log you in as 'withnail@nature.com'"

This definition language is called ‘Guerkin’ and provides a simple, unambiguous manner with which to define a feature, or requirement, of the system. Cucumber allows us take these plain text features descriptions and turn them in to automated tests that ensure a system fulfills the business requirements.

Features

A feature is a discrete chunk of functionality; above, authentication is a good example. Features should not have too wide a remit; “Provide a blogging system” would be too high level. Better would be defining features such as “Adding a comment to a blog post”, etc. Features are comprised of a header, and a series of scenarios.

Feature header

A feature is introduced with a simple header that defines the business benefit for implementing it. You define a feature by saying:

In order to…

“publicise my scientific event to a wide community of scientists, researchers and journalists”

Defines the business benefit to the user of delivering the feature.

As a…

“event organiser”

The role definition; specifies who is the end user wanting to achieve the benefit.

I want to…

“register as an event organiser on events.nature.com”

What the user has to do in order to achieve the perceived benefit.

Scenarios

Each feature is likely to have a number of scenarios. Using authentication as an example, there are several possible user scenarios we need to test. Using unknown logon, supplying the wrong password, logging in correctly, etc. These can all be defined as separate scenarios using Guerkin.

A scenario is defined in terms of steps called given, when or then, and can be comprised of any number of those steps in any order.

Given…

A given is a pre-requisite for a user-led action taking place. In order to test a user logging in for instance, an account for that user has to exist. If multiple Givens are required then you use the And term to link them. I.e:

  Given I have an account with email "bob@nature.com" with the password "letmein"
    And the account "bob@nature.com" has 3 associated comments

When…

A when is typically a user action, or more likely a series of user actions. For web-applications these would be things like clicking on links/buttons, filling in text fields, selecting items from a list, etc.

  When I fill in "email" with "bob@nature.com"
    And I fill in "password" with "letmein"
    And I press "Log in"

By default, Cucumber uses a library called Webrat to carry out the actions.

Then…

A then defines an outcome; an expected side-effect of the previous actions. Then steps are usually concerned with ensuring the user sees something on a screen, or that an entity has been created in the database.

  Then I should see "Welcome to Nature.com"
    And I should have a new session

The nice thing about defining scenarios using these simple building blocks is that we can build up complex user stories with ease:

  Scenario: I want to post a free event
  Given the following users: 
    | username | password | enabled |
    | i@nature.com | itsasecret | true |
    And I am logged in as "i@nature.com"
    And I view the homepage
  When I follow "Post an event"
  Then I should see "Please enter your event details"
  When I fill in the new event form
    And I press "Post Event"
  Then I should see "Your event has been submitted for approval"
    And I should be redirected to the event screen
    And there should be a new event
    And an emails should be sent to "i@nature.com" and "events-admin@nature.com"

As was mentioned before, we can use multiple When/Then steps in order to tell a much longer and more compelling user story.

It is worth mentioning that at this point Cucumber does not allow you to “include” a reusable collection of steps in to a scenario; for instance, if you wanted to include a standard set of login steps in to multiple scenarios you’d either have to cut/paste (not very DRY) or define a “super-step” that undertakes them all for you.

Making Guerkin sing

Now Guerkin is very nice and everything, but the true power of Cucumber is to take those feature definitions and turn them in to executable integration tests. You do this by taking each step definition in your features and writing a matching statement in one of your step_definition.rb files. For instance, given a step:

  Given I have an account with email "bob@nature.com" and password "letmein"

You would typically create a file under the step_definitions directory (see Setting Cucumber up) called user_steps.rb, and in it would define something like the following:

  Given /I have an account with email "(.*)" and password "(.*)"/ do |email, password|
    User.create(:email => email, :password => password)
  end

As you can see, the step matcher can parameterise information in the step using standard ruby regexp matchers, and these are passed as arguments to the step block. In our simple example the code creates the user referred to in the step so it’s available to subsequent steps.

Using Webrat in your steps

Webrat provides a way of simulating user actions on the browser such as following links, clicking on buttons and entering text in to fields. Webrat is maintained at github.

When you set Cucumber up with Webrat a set of default step definitions is provided in features/step_definitions/webrat_steps.rb. This gives you the ability to do things like:

  When I press "Submit"

Which would be mapped to

  When /^I press "(.*)"$/ do |button|
    click_button(button)
  end

click_button is a webrat method. Others include follow_link, fill_in and select. You can use these methods in your own actions too. For instance, when defining a scenario including completing a lengthy form, rather than defining each field as a separate step using the webrat_steps helpers, it’s better to define your own “When I complete the form” step, then have:

  When /^I complete the form$/ do
    fill_in "firstname", :with => "Bob"
    fill_in "lastname", :with => "Bob"
    select "mr", :from => "title"
  end

Much neater.

Using tables

Another useful Cucumber construct is the table, also known as a Fixture Input Table (FIT). These enable bulk creation of test data for the scenario. I’ve already shown the use of these, in fact in the original example:

  ...
  Scenario: Log in with invalid login
    Given the following users exist: 
      | email | password | first_name | last_name | title | country |
      | i@nature.com | itsasecret | Bob | Smith | mr | UK |
  ...

The given step here has a FIT table immediately afterwards. These are simple to define; the first row specifies the attributes to pass, and subsequent rows specify the values for each instance you want to create. The companion step definition might look something like:

Given /the following users exist:/ do |user_table|
  user_table.hashes.each do |attrs|
    User.create(attrs)
  end
end

Each row in the FIT table is turned in to a Ruby hash that maps the column name to the value. These hashes can be used in ActiveRecord object creators as shown above.

FIT tables can also be used in scenario outlines.

Setting Cucumber up

This section tells you how to get your Rails project running with Cucumber.

How do I get it.

The project is maintained on github. To install the gem do the following:

  • If you have not already done so, add github.com to your gem sources:
      gem sources -a http://gems.github.com
      
  • Now install the gem:
      sudo gem install aslakhellesoy-cucumber
      
  • It’s also worth installing webrat (for defining browser actions in your Cucumber features) and rspec (for your unit/functional testing)
      sudo gem install webrat rspec rspec-rails
      

Setting your Rails app up to use Cucumber

Simply go to your Rails root directory and run:

  ruby script/generate cucumber
  

This creates the following structure under your Rails project:

  rails-app/
    app/
    ...
    features/
      example.feature
      step_definitions/
      support/
        env.rb
  

Your plain text feature definitions go in the features directory. The step_definitions directory contains the code to map your feature steps to executable test code, and support typically just contains the env.rb file, roughly equivalent to the rspec spec_helper.rb file, or your Rails environment.rb file.

I want more!

This is really a light introduction to Cucumber. You will find a lot more information at the Cucumber Wiki.

Written by spanx

17th July, 2009 at 7:31 pm

Posted in Code

Tagged with , ,

Welcome!

with one comment

I aim to actually keep this blog alive. I’ll be posting about stuff I’m doing at Nature Publishing Group (where I work as a senior developer in the new products team), and stuff I’m doing for Indmill.com (the music start-up I part-own and run).

It’ll be a mix of things about coding, music, random interesting things and probably a few pictures and anecdotes about my kids doing crazy/amazing/funny things. So settle down, pour a glass of red and soak it up.

Thanks, Steve

Written by spanx

17th July, 2009 at 7:05 pm

Posted in Uncategorized