When I started playing around with Ember.js almost a year ago, the testability story left something to be desired. You could unit test an object without any trouble, but a unit test is only one way to get feedback when you’re building a software product. In addition to unit tests, I wanted a way to verify the integration of multiple components. So like most people testing rich JavaScript applications, I reached for the mother of all testing tools, Selenium.

Now before I bash it, without a proper introduction, it’s worth mentioning that Selenium is a great way to verify your entire web application works with a full production-like database and all your production dependencies, etc. And from a QA perspective, this tool can be a great resource for teams who need end-to-end UI acceptance tests.

But over time, a seemingly small test suite built on Selenium can begin to drag the velocity of your team to a snails pace. One easy way to reduce this pain is to avoid building a large application in the first place. If you build a handful of smaller web applications instead, it might help keep you afloat for a little longer because no individual build will crush the team, as you grow.

But even on a small project, the real problem with Selenium is that it’s not part of the test driven development process. When I’m doing red/ green/ refactor I don’t have time for slow feedback in any form. I needed a way to write both unit and integration tests that would provide quick feedback to help me shape the software I was writing in a more iterative way. If you are using a version of Ember.js >= RC3, you’re in luck because writing a unit or integration test is a walk in the part.

Installing the Test Runner

Now that we can write JavaScript tests for our application, how do we execute them? Most developers start out using the browser directly, but because I wanted something I could execute headless from the command line in a CI environment with a rich ecosystem full of plugins, I looked to Karma.

What I liked about Karma is that it only wants to be your test runner. It doesn’t care what JavaScript test framework you use or what client side MVC framework you use. It’s simple to get started with and writing tests that execute against your production Ember.js application is just a few lines of configuration.

But before we can configure Karma, we need to install it using npm. I recommend installing it locally so you can keep your npm modules isolated per project. To do this, add a file named package.json‘ to the root of your project that looks something like the below.

{
  "dependencies": {
    "karma-qunit": "*",
    "karma": "0.10.2"
  }
}

This example will require both Karma, and a plugin for QUnit. After you save the package.json file above, drop back to the command line and type npm install to pull down the required Node modules.

After the npm install completes, you will now see a new folder with the name node_modules in the root of your project. This folder contains all the JavaScript code we just pulled down with npm, including Karma and the QUnit plugin. If you drill down even further to node_modules/karma/bin/ you will see the Karma executable. We will be using this to configure the test runner, execute tests from the command line, etc.

Configure the Test Runner

Next we need to configure karma so it knows how to execute the QUnit tests. Type karma init from the root of the project. You will be prompted with a list of questions. The first will ask what testing framework you want to use, hit Tab until you see qunit, then hit Enter. Next answer no to the Require.js question, as we won’t be using it for this sample application. Tab until you see PhantomJS for the third question and you will need to hit Enter twice as it allows multiple options here. As for the rest, just leave them at their default option.

When you are done, you should see Karma has generated a configuration file named karma.conf.js in the root or your project. If you want to read more about the various options Karma supports, you might find the comments helpful. For the sake of this example, I have a simplified version of the configuration file to keep things beginner friendly.

If you want to follow along, delete the generated configuration file and replace it with this one.

module.exports = function(karma) {
  karma.set({
    basePath: 'js',
     
    files: [
      "vendor/jquery/jquery.min.js",
      "vendor/handlebars/handlebars.js",
      "vendor/ember/ember.js",
      "vendor/jquery-mockjax/jquery.mockjax.js",
      "app.js",
      "tests/*.js"
    ],
     
    logLevel: karma.LOG_ERROR,
    browsers: ['PhantomJS'],
    singleRun: true,
    autoWatch: false,
      
    frameworks: ["qunit"]
  });
};

This should be fairly similar to what Karma generated earlier, I’ve just removed all the comments and cut out a few options we don’t care about right now. In order to write the first unit test, I had to tell Karma a little more about the project structure.

At the top of the configuration file, you will see that I’ve set the basePath to js because all of the JavaScript assets live under this folder in the project. Next, I told Karma where it can find the JavaScript files required to test our simple application. This includes jQuery, Handlebars, Ember.js and the app.js file itself.

Writing the First Unit Test

Now we can add the first unit test file to the project. First make a new folder named tests and nest it under the js folder. Add a file in this new directory named unit_tests.js that looks something like this.

test('hello world', function() {
  equal(1, 1, "");
});

This test isn’t doing anything valuable yet, but it will help us verify we have everything wired up with Karma to execute it correctly. Notice in the Karma files section, we already added the js/tests directory. This way Karma will pull in every JavaScript file we use to test our application with, going forward.

Now that we have Karma configured correctly, execute the qunit tests from the command line using ./node_modules/karma/bin/karma start.

If you have everything setup correctly, you should see Karma execute one test and it being successful. To verify it executed the test we just wrote, go make it fail by altering the equals statement. For example, you could do the following:

test('hello world', function() {
  equal(1, 2, "boom");
});

If you can fail this and make it pass again, it’s time to write a test with a little more purpose.

The Sample Application

But before we get started, lets discuss the sample application used throughout this post. In the screenshot below, you see we have a very simple grid of users. In the HTML table, each user is shown by first name along with a button to delete that user. At the top of the application you will see an input for the first name, last name and finally a button that will add another user to the table when clicked.

https://dl.dropboxusercontent.com/u/716525/content/images/2013/pre-tuts.png

The example application has three problems. First, we want to show the user’s first and last name, not just the first name. Next, when you click a delete button it won’t actually remove the user. And finally, when you add a first name, last name and click add, it won’t put another user into the table.

On the surface, the full name change appears to be the simplest. It also turned out to be a great example that shows when you should write a unit test, an integration test or both. In this example, the quickest way to get feedback is to write a simple unit test that asserts the model has a computed property fullName.

Unit Testing the Computed Property

Unit testing an ember object is easy, you simply create a new instance of the object and ask for the fullName value.

test('fullName property returns both first and last', function() {
  var person = App.Person.create({firstName: 'toran', lastName: 'billups'});
  var result = person.get('fullName');
  equal(result, 'toran billups', "fullName was " + result);
});

Next if you go back to the command line and run ./node_modules/karma/bin/karma start, it should show one failing test with a helpful message describing fullName as undefined currently. To fix this, we need to open the app.js file and add a computed property to the model that returns a string of the combined first and last name values.

App.Person = Ember.Object.extend({
  firstName: '',
  lastName: '',
  fullName: function() {
    var firstName = this.get('firstName');
    var lastName = this.get('lastName');
    return firstName + ' ' + lastName;
  }.property()
});

If you drop back to the command line and run ./node_modules/karma/bin/karma start you should now see a passing unit test. You can extend this example by writing a few other unit tests to show that the computed property should change when either the first or last name is updated on the model.

test('fullName property returns both first and last', function() {
  var person = App.Person.create({firstName: 'toran', lastName: 'billups'});
  var result = person.get('fullName');
  equal(result, 'toran billups', "fullName was " + result);
});

test('fullName property updates when firstName is changed', function() {
  var person = App.Person.create({firstName: 'toran', lastName: 'billups'});
  var result = person.get('fullName');
  equal(result, 'toran billups', "fullName was " + result);
  person.set('firstName', 'wat');
  result = person.get('fullName');
  equal(result, 'wat billups', "fullName was " + result);
});

test('fullName property updates when lastName is changed', function() {
  var person = App.Person.create({firstName: 'toran', lastName: 'billups'});
  var result = person.get('fullName');
  equal(result, 'toran billups', "fullName was " + result);
  person.set('lastName', 'tbozz');
  result = person.get('fullName');
  equal(result, 'toran tbozz', "fullName was " + result);
});

If you add these two additional tests and run all three from the command line, you should have two failing. To get all three tests passing, modify the computed property to listen for changes on both the first name and last name. Now if you run ./node_modules/karma/bin/karma start from the command line, you should have three passing tests.

App.Person = Ember.Object.extend({
  firstName: '',
  lastName: '',
  fullName: function() {
    var firstName = this.get('firstName');
    var lastName = this.get('lastName');
    return firstName + ' ' + lastName;
  }.property('firstName', 'lastName')
});

Add the Karma-Ember-Preprocessor and Configure It

Now that we have a computed property on the model, we need to look at the template itself because currently we don’t use the new fullName property. In the past, you would need to wire up everything yourself, or use Selenium to verify the template gets rendered correctly. But with ember-testing you can now integration test this by adding a few lines of JavaScript and a plugin for Karma.

First open the package.json file and add the karma-ember-preprocessor dependency. After you update the package.json file, do npm install from the command line to pull this down.

{
  "dependencies": {
    "karma-ember-preprocessor": "*",
    "karma-qunit": "*",
    "karma": "0.10.2"
  }
}

Now that you have the pre-processor installed, we need to make Karma aware of the template files. In the files section of your karma.conf.js file add the following to tell Karma about the Handlebars templates.

module.exports = function(karma) {
  karma.set({
    basePath: 'js',
    
    files: [
      "vendor/jquery/jquery.min.js",
      "vendor/handlebars/handlebars.js",
      "vendor/ember/ember.js",
      "vendor/jquery-mockjax/jquery.mockjax.js",
      "app.js",
      "tests/*.js",
      "templates/*.handlebars"
    ],
    
    logLevel: karma.LOG_ERROR,
    browsers: ['PhantomJS'],
    singleRun: true,
    autoWatch: false,
     
    frameworks: ["qunit"]
  });
};

Next we need to tell Karma what to do with these handlebars files, because technically we want to have each template precompiled before it’s handed over to PhantomJS. Add the preprocessor configuration and point anything with a file extension of *.handlebars at the ember preprocessor. Also you need to add the plugins configuration to register the ember pre-processor (along with a few others that normally get included with Karma’s default configuration).

module.exports = function(karma) {
  karma.set({
    basePath: 'js',
     
    files: [
      "vendor/jquery/jquery.min.js",
      "vendor/handlebars/handlebars.js",
      "vendor/ember/ember.js",
      "vendor/jquery-mockjax/jquery.mockjax.js",
      "app.js",
      "tests/*.js",
      "templates/*.handlebars"
    ],
     
    logLevel: karma.LOG_ERROR,
    browsers: ['PhantomJS'],
    singleRun: true,
    autoWatch: false,
     
    frameworks: ["qunit"],
     
    plugins: [
      'karma-qunit',
      'karma-chrome-launcher',
      'karma-ember-preprocessor',
      'karma-phantomjs-launcher'
    ],
     
    preprocessors: {
      "**/*.handlebars": 'ember'
    }
  });
};

Integration Testing the Data-Bound Template

Now that we have the Karma configuration setup for integration testing, add a new file named integration_tests.js under the tests folder. Inside this folder we need to add a simple test to prove we can stand up the entire Ember.js application without error. Add a simple qunit test to see if we can hit the '/' route and get the basic HTML returned. For the initial test, we are only asserting that the table tag exists in the HTML that was generated.

test('hello world', function() {
  App.reset();
  visit("/").then(function() {
    ok(exists("table"));
  });
});

Notice we are using a few helpers that are built into ember-testing like visit and find. The visit helper is an ember friendly way of telling the application what state to be at during the execution. This test starts at the '/' route because that is where the People models get bound to the template and our HTML table is generated. The find helper is a quick way to lookup elements in the DOM using CSS selectors like you would with jQuery to verify something about the markup.

Before we can run this test we need to add a test helper file that will inject the test helpers and set a generic root element. Add the code below, to a file named integration_test_helper.js in the same tests directory. This will ensure our application has the test helpers at execution time.

document.write('<div id="ember-testing-container"><div id="ember-testing"></div></div>');
 
App.rootElement = '#ember-testing';
App.setupForTesting();
App.injectTestHelpers();

function exists(selector) {
  return !!find(selector).length;
}

Now from the command line you should be able to execute the integration test above. If you got a passing test, remove the table from the handlebars template to make it fail (just to help prove Ember was generating the HTML using that template).

Now that we have the integration tests setup, it’s time to write the one that asserts we show each user’s fullName instead of their firstName. We want to first assert that we get two rows, one for each person.

test('hello world', function() {
  App.reset();
  visit("/").then(function() {
    var rows = find("table tr").length;
    equal(rows, 2, rows);
  });
});

Note: The application is currently returning hard coded data to keep everything simple at the moment. If you are curious why we get two people, here is the find method on the model:

App.Person.reopenClass({
  people: [],
  find: function() {
    var first = App.Person.create({firstName: 'x', lastName: 'y'});
    var last = App.Person.create({firstName: 'x', lastName: 'y'});
    this.people.pushObject(first);
    this.people.pushObject(last);
    return this.people;
  }
});

If we run the tests now, we should still have everything passing because two people are returned as we would expect. Next, we need to get the table cell that shows the person’s name and assert it’s using the fullName property instead of just firstName.

test('hello world', function() {
  App.reset();
  visit("/").then(function() {
    var rows = find("table tr").length;
    equal(rows, 2, rows);
    var fullName = find("table tr:eq(0) td:eq(0)").text();
    equal(fullName, "x y", "the first table row had fullName: " + fullName);
  });
});

If you run the above test you should see a failing test because we haven’t yet updated the template to use fullName. Now that we have a failing test, update the template to use fullName and run the tests using ./node_modules/karma/bin/karma start. You should now have a passing suite of both unit and integration tests.

Should I Write Unit or Integration Tests?

If you are asking yourself, “when should I write a unit test vs. an integration test?”, the answer is simply: what will be less painful? If writing a unit test is faster and it explains the problem better than a much larger integration test, then I say write the unit test. If the unit tests seem less valuable because you are doing basic CRUD and the real behavior is in the interaction between components, I say write the integration test. Because the integration tests written with ember-testing are blazingly fast, they are part of the developer feedback cycle and should be used similarly to a unit test when it makes sense.

To show a CRUD like integration test in action, write the following test to prove the add button puts the person into the collection and that a new row gets rendered in the handlebars template.

test('add will append another person to the html table', function() {
  App.Person.people = [];
  App.reset();
  visit("/").then(function() {
    var rows = find("table tr").length
    equal(rows, 2, "the table had " + rows + " rows");
    fillIn(".firstName", "foo");
    fillIn(".lastName", "bar");
    return click(".submit");
  }).then(function() {
    equal(find("table tr").length, 3, "the table of people was not complete");
    equal(find("table tr:eq(2) td:eq(0)").text(), "foo bar", "the fullName for the person was incorrect");
  });
});

Start by telling the test what state you want to work with, then using the fillIn helper, add a first name and last name. Now if you click the submit button it should add that person to the HTML table, so in the returning then we can assert that three people exist in the HTML table. Run this test and it should fail because the Ember controller isn’t complete.

To get the test passing, add the following line to the PeopleController

App.PeopleController = Ember.ArrayController.extend({
  actions: {
    addPerson: function() {
      var person = {
        firstName: this.get('firstName'),
        lastName: this.get('lastName')
      };
      App.Person.add(person);
    }
  }
});

Now if you run the tests using ./node_modules/karma/bin/karma start it should show three people in the rendered HTML.

The last test is the delete, notice we find the button for a specific row and click it. In the following then we simply verify one less person is shown in the HTML table.

test('delete will remove the person for a given row', function() {
  App.Person.people = [];
  App.reset();
  visit("/").then(function() {
    var rows = find("table tr").length;
    equal(rows, 2, "the table had " + rows + " rows");
    return click("table .delete:first");
  }).then(function() {
    equal(find("table tr").length, 1, "the table of people was not complete
  });
});")})})

To get this passing, simply add the following line to the PeopleController:

App.PeopleController = Ember.ArrayController.extend({
  actions: {
    addPerson: function() {
      var person = {
        firstName: this.get('firstName'),
        lastName: this.get('lastName')
      };
      App.Person.add(person);
    },
    deletePerson: function(person) {
      App.Person.remove(person);
    }
  }
});

Run the tests from the command line and you should once again have a passing suite of tests.

Conclusion

So that wraps up our sample application. Feel free to ask any questions down in the comments.

Bonus: But I’m Already Using Grunt…

If you prefer to use Grunt instead of the karma-ember-preprocessor, simply remove the plugins and preprocessors configuration. Also remove templates/*.handlebars from the files section as Karma won’t need to precompile the templates. Here is a simplified karma.conf.js that works when using grunt to precompile the handlebars templates.

module.exports = function(karma) {
  karma.set({
    basePath: 'js',
 
    files: [
      "lib/deps.min.js", //built by your grunt task
      "tests/*.js"
    ],
    
    logLevel: karma.LOG_ERROR,
    browsers: ['PhantomJS'],
    singleRun: true,
    autoWatch: false,
     
    frameworks: ["qunit"]
  });
};

And that’s it!