A better method of handling multistep forms in rails

If there ever was a task that seemed straightforward from the outside, but then turned out to be really complicated when you got into it, a multi-step form is it.

Why are multi-step forms so hard? The main challenge seems to be validating partial objects.

There is not necessarily a one-to-one relationship between a step in your multi-step form and a model in your application. For example, let's say you have a user model with attributes email, password, first_name, last_name and homepage_url (all required). To avoid intimidating the user with a long form, you put email and password on the first page of your multi-step form and the rest of the fields on the second page. What you have is a single model split across two forms. (In reality you should maybe make a distinction between a user profile and a user account, but that's another matter.)

This splitting of models across forms means you have to ask awkward questions like, "Hey, ActiveRecord, is the half of this object I have so far valid?" I don't think ActiveRecord was designed for validating parts of objects. From the examples I've seen where people try to do it, things get yucky

Partial object validation is hard, and there's also a bigger problem

Before you go on, please read (or at least scan) Building Partial Objects Step by Step in the Wicked gem wiki. The Wicked gem is absolutely the best Rails multi-step form tool I could find, and it looks like a lot of care was taken to cross all the t's and dot all the i's. I'm very impressed by the gem. That said, I have a different idea to suggest on how multi-step forms could be handled.

You understand the MVC architecture pattern. It's an example of layered application architecture, and in MVC there are of course three layers. What you might not know is that it's possible, and often wise, to divide your application into finer layers than that. Some of those finer layer include:

Domain layer: This is logic that exists independently of the fact that you're writing a computer program. It's just part of the domain with which you're working. Example: calculating the length and cost of an appointment.
Persistence layer: The persistence layer has to do with, as you might have guessed, persistence. Example: saving an appointment to the database.
Application layer: This is the administrative, nuts-and-bolts kind of work. Example: sending a thank-you email to the client.

Most developers, including myself, are used to mixing domain, persistence, and application layers all together, willy-nilly, in the model layer. And for most small and medium-sized jobs, this is totally good and appropriate, the same way you wouldn't fire up an MVC framework to write tiny one-off throwaway script. But the hairier the task, the more layering is called for.

What I'm saying is that I think multi-step forms are too hairy a task to put it all in the model, view and controller layers. Here are some of the responsibilities of a multi-step form and the layers in which I think they belong:

Responsibility belongs to Layer:

Validating a single step: Domain
Moving forward and backward among steps: Application
Saving data to the database: Persistence

The Wicked gem—which, for the record, I think is a great accomplishment and way more work than I've done toward multi-step forms—puts a large amount of the logic in your ActiveRecord model class, which I think is a paradigm that could maybe be improved

My proposed solution, but first, a counterexample

This example, from the Wicked gem, is what it looks like when you take the fact that you have a multi-step form for creating a Product and you let that fact influence your model class definition:

class Product < ActiveRecord::Base validates :name, :presence => true, :if => :active_or_name?
  validates :price, :presence => true, :if => :active_or_price?
  validates :category, :presence => true, :if => :active_or_category?

  def active?
    status == 'active'
  end

  def active_or_name?
    status.include?('name') || active?
  end

  def active_or_price?
    status.include?('price') || active?
  end

  def active_or_category?
    status.include?('category') || active?
  end
end

Again, no offense to Wicked, but all those conditional validations really bother me. Plus it seems like the fact that you have a multi-step form in your application shouldn't be a concern of your model layer. Why should your models care how the data gets

My brilliant idea

So, finally, my idea is that instead of this:

Validate step 1 of the form
Save step 1 to the database, move onto the next step
Validate step 2 of the form
Save step 2 to the database, move onto the next step
etc.
Done

We should do this:

Validate step 1 of the form, collect the data, move onto the next step
Validate step 2 of the form, collect the data, move onto the next step
etc.
Save all the collected data to the database
Done

Let me put it another way as well: instead of building up an entity one piece at a time and permanently saving each piece as we go, we build up an entity candidate and then, if and when that candidate is valid and complete, the candidate entity becomes a true instance and we save it to the database.

And as you might have guessed, the mechanism that guides the user through the form steps is a separate mechanism from what saves everything to the database. Imagine something like this:

product_candidate = ProductCandidate.new
# Validate each step, collect data, etc.
Product.create_from_candidate!(product_candidate)

That way, not only do you have presumably cleaner code, but you also don't have to subject your database to inconsistent data. You only persist your data once you have a complete entity (or set of entities).

A rudimentary implementation

I've actually implemented a very rudimentary version of this idea. It's in fact so rudimentary that the only interface it has is through the console, but it's still hopefully a useful illustration.

First I instantiate my profile_candidate:

> profile_candidate = ProfileCandidate.new(ProfileCreationProcess.new)

Don't worry for now about what ProfileCreationProcess is. An EntityCandidate (from which ProfileCandidate inherits) knows certain things about itself, like which step it's on, whether it's complete and whether it's valid.

> profile_candidate.current_step_number
=> 0
> profile_candidate.complete?
=> false
> profile_candidate.valid?
=> false

By the way, an EntityCandidate determines whether it's complete based on the current step vs. total number of steps, which is something else it knows:

> profile_candidate.total_number_of_steps
=> 3

The three steps in this case are three super simple ones: a "form" containing first_name, a "form" with email and a third and final "form" containing phone. Here we can see what it's like to complete a step:

> profile_candidate.first_name = 'Jason'
=> "Jason"
> profile_candidate.valid?
=> true
> profile_candidate.complete?
=> false
> profile_candidate.save
=> true
> profile_candidate.current_step_number
=> 1

If we go ahead and complete the next two steps (and EntityCandidate is smart enough to "catch up" if we go multiple steps without saving), you'll see that at the end our profile_candidate is complete:

> profile_candidate.email = 'jason@benfranklinlabs.com'
=> "jason@benfranklinlabs.com"
> profile_candidate.phone = '(616) 856-8075'
=> "(616) 856-8075"
> profile_candidate.save
=> true
> profile_candidate.current_step_number
=> 3
> profile_candidate.complete?
=> true

The code

The code for ProfileCandidate is really simple:

class ProfileCandidate < EntityCandidate
  attr_accessor :first_name, :email, :phone, :creation_process, :completed_steps
end

EntityCandidate has a little more to it:

class EntityCandidate
  def initialize(creation_process)
    @creation_process = creation_process
    @completed_steps = []
  end

  def valid?(step_number = current_step_number)
    step = @creation_process.steps[step_number]
    step.valid?(send(step.field_name))
  end

  def complete?
    @completed_steps.length == total_number_of_steps
  end

  def last_completed_step_number
    @completed_steps.last || -1
  end

  def current_step_number
    last_completed_step_number + 1
  end

  def total_number_of_steps
    @creation_process.steps.length
  end

  def save
    if save_step(current_step_number)
      try_saving_any_later_steps
      true
    else
      false
    end
  end

  def save_step(step_number)
    return false unless valid?(step_number)
    @completed_steps << step_number
    true
  end

  def try_saving_any_later_steps
    step_number = current_step_number
    while step_number < total_number_of_steps do
      save_step(step_number)
      step_number += 1
    end
  end
end

Finally, here are ProfileCreationProcess and ProfileCreationStep:

class ProfileCreationProcess
  attr_accessor :steps

  def initialize
  @steps = []

  # This validation is *extremely* rudimentary!
  step :first_name, Proc.new { |first_name| first_name.to_s != "" }
  step :email, Proc.new { |email| email.to_s != "" }
  step :phone, Proc.new { |phone| phone.to_s != "" }
  end

  def step(field_name, validator)
    @steps << ProfileCreationStep.new(field_name, validator)
  end
end

class ProfileCreationStep
  attr_accessor :field_name

  def initialize(field_name, validator)
    @field_name = field_name
    @validator = validator
  end

  def invalid?(value)
    !valid?(value)
  end

  def valid?(value)
    @validator.call(value)
  end
end

There are, of course, some problems with my implementation

I haven't yet devised a way to wire it up to an actual form
My validations are comically rudimentary
I don't have a way to move backward through the process
I don't have a way to save a partially-completed form
I don't have a way to save the entity candidate at all
So I don't exactly have a gem packaged up and ready to go for you. But I do think the idea is a pretty solid one: build up an entity candidate, then only once we've validated that complete candidate do we swear that candidate into the database—and we do so in a way that appropriately separates our application layers.

What do you think?

What do you think of this idea? Can you poke holes in it? Is the whole thing stupid? Is it the best idea ever? Would you like to see a gem? Tweet me at @jasonswett or email me at jason@benfranklinlabs.com with your thoughts.