Unit testing in Javascript, especially with RequireJS can be a bit of challenge. Jasmine, which is our unit testing framework does not have any out of the box support for RequireJS. I have seen a few ways of integrating RequireJS but that requires hacking the SpecRunner.html file, the main test harness that executes all jasmine tests. That wasn’t really an option for us as we were using a ruby gem called jasmine to auto generate this html file from our spec files. There is however an experimental gem created by Brendan Jerwin that provides RequireJS integration. We did consider that option before ruling it out for lack of official support. After a bit of flailing around, we finally hit upon a little nugget in the core jasmine framework that seemed to provide a solution.

Async tests in Jasmine

For a long time, most of our tests used the standard prescribed procedure in jasmine, which is describe() with a bunch of it()s. This worked well for the most part until we switched to RequireJS as our script loader. Then there was only blood red on our test pages.

Clearly jasmine and RequireJS have no mutual contract, but there is a way to run async tests in jasmine with methods like runs(), waits() and waitsFor(). Out of these, runs() and waitsFor() were the real nuggets, which complement each other when running async tests.

waitsFor() takes in a function that should return a boolean when the work item has completed. Jasmine will keep calling this function until it returns true, with a default timeout of 5 seconds. If the worker function doesn’t complete by that time, the test will be marked as a failure. You can change the error message and the timeout period by passing in additional arguments to waitsFor().

runs() takes in a function that is called whenever it is ready. If a runs() is preceded by a waitsFor(), it will execute only when the waitsFor() has completed. This is great since it is exactly what we need to make our RequireJS based tests to run correctly. In code, the usage of waitsFor() and runs() looks as shown below. Note that I am using CoffeeScript here for easier readability.

— Short CoffeeScript Primer —
In CoffeeScript, the -> (arrow operator) translates to a function(){} block. Functions can be invoked without the parenthesis,eg: foo args is similar to foo(args). The last statement of a function is considered as the return value. Thus, () -> 100 would become function(){ return 100; } “With this primer, you should be able to follow the code snippet below.”

1
2
3
4
5
6
7

    it "should do something nice", ->
        waitsFor ->
          isWorkCompleted()

        runs ->
            completedWork().doSomethingNice()
  

Jasmine meets RequireJS

waitsFor() along with runs() holds the key to running our RequireJS based tests. Within waitsFor() we wait for the RequireJS modules to load and return true whenever those modules are available. In runs() we take those modules and execute our test code. Since this pattern of writing tests was becoming so common, I decided to capture that into a helper method, called ait().

1
2
3
4
5
6
7
8
9
10

ait = (description, modules, testFn)->
    it description, ->
        readyModules = []
        waitsFor ->
            require modules, -> readyModules = arguments
            readyModules.length is modules.length # return true only if all modules are ready

        runs ->
            arrayOfModules = Array.prototype.slice.call readyModules
            testFn(arrayOfModules...)

If are wondering why the name ait(), it is just to keep up with the spirit of jasmine methods like it for the test case and xit for ignored test case. Hence ait, which stands for “async it”. This method takes care of waiting for the RequireJS modules to load (which are passed in the modules argument) and then proceeding with the call to the testFn in runs(), which has the real test code. The testFn takes the modules as individual arguments. Note the special CoffeeScript syntax arrayOfModules... for the expansion of an array into individual elements.

The ait method really reads as: it waitsFor() the RequireJS modules to load and then runs() the test code

To make things a little clear, here is an example usage:

1
2
3
4
5
6
7

describe 'My obedient Model', ->

    ait 'should do something nice', ['obedient_model', 'sub_model'], (ObedientModel, SubModel)->
        subModel = new SubModel
        model = new ObedientModel(subModel)
        expect(model.doSomethingNice()).toEqual "Just did something really nice!"
      

The test case should do something nice, takes in two modules: obedient_model and sub_model, which resolve to the arguments: ObedientModel and SubModel, and then executes the test code. Note that I am relying on the default timeout for the waitsFor() method. So far this works great, but that may change as we build up more tests.

In the world of jQuery or for that matter, any JavaScript library, callbacks are the norm for programming asynchronous tasks. When you have several operations dependent on the completion of some other operation, it is best to handle them as a callback. At a later point when your dependent task completes, all of the registered callbacks will be triggered.

This is a simple and effective model and works great for UI applications. With jQuery.Deferred(), this programming model has been codified with a set of utility methods.

$.Deferred() is the entry point for dealing with deferred operations. It creates a “promise” (a.k.a Deferred object) to trigger all the registered done() or then() callbacks once the Deferred object goes into the resolve() state. This is according to the CommonJS specification for Promises. I am not going to cover all the details of $.Deferred(), since the jQuery docs do a much better job. Instead, I’ll jump right into the main topic of this post.

The soup of AMD, $.Deferred and Google Maps

In one of my recent explorations with web apps, the AMD pattern turned out to be extremely useful. AMD, with the RequireJS library, forces a certain structure on your project and makes building large web apps more digestible. Abstractions like the require/define calls allows building apps that are more composable and extensible. It sure is a great way to think about composable JS apps in contrast to the crude <script> tags.

With these abstractions, it was easier to think of the app as a set of modules. Some modules provide base level services, while others depend on such service-modules. One particular module, which also happens to be the entry point into the app, was heavily dependent on the Google Maps API. Early on, it was decided to never keep the user waiting for the maps to load and allow interaction right from the get go.This meant that they could do some map-related tasks even before the maps API had loaded. Although this felt impossible at the onset, it turned out to be quite easy, all thanks to $.Deferred().

The first step was to wrap the Google Maps API in a GoogleMaps object. This hides away the details about loading the maps while allowing the user to carry on with the map related tasks.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

function GoogleMaps() {
  
}

GoogleMaps.prototype.init = function() {
  
};

GoogleMaps.prototype.createMap = function(container) {
};

GoogleMaps.prototype.search = function(searchText) {
};

GoogleMaps.prototype.placeMarker = function(options) {
};

The calls to createMap, search and placeMarker need to be queued up until the maps API has loaded. We start off with a single $.Deferred() object, _mapsLoaded

1
2
3
4
5

_mapsLoaded = $.Deferred()

function GoogleMaps() {
  // ...
}

Then in each of the methods mentioned earlier, we wrap the actual code inside a deferred.done(), like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

function GoogleMaps() {
    _mapsLoaded.done(_.bind(function() {
        this.init();
    }, this));
}

GoogleMaps.prototype.init = function() {
};

GoogleMaps.prototype.createMap = function(container) {
    _mapsLoaded.done(_.bind(function() {
      // create the maps object
    }, this));
};

GoogleMaps.prototype.search = function(searchText) {
    _mapsLoaded.done(_.bind(function() {
      // search address
    }, this));
};

GoogleMaps.prototype.placeMarker = function(options) {
    _mapsLoaded.done(_.bind(function() {
      // position marker
    }, this));
};
  

With this, we can continue making calls to each of these methods as if the maps API is already loaded. Each time we make a call, it will be pushed into the deferred queue. At some point, when the maps API is loaded, we need to call a resolve() on the deferred object. This will cause the queue of calls to be flushed and resulting in real work being done.

One aside on the code above is the use of _.bind(function(){}, this)_. This is required because the callback to done() changes the context of this. To keep it pointing to the GoogleMaps instance, we employ _.bind().

1
2
3
4
5
6
7

window.gmapsLoaded = function() {
    delete window.gmapsLoaded;
    _mapsLoaded.resolve();
};

require(['http://maps.googleapis.com/maps/api/js?sensor=true&callback=gmapsLoaded']);
  

The google maps API has an async loading option with a callback name specified in the query parameter for the api URL. When the api loads, it will call this function (in our case: gmapsLoaded). Note that this needs to be a global function, ie. on the window object. A require call (from RequireJS) makes it easy to load this script.

Once the callback is made, we finally call resolve() on our deferred object: _mapsLoaded. This will trigger the enqueued calls and the user starts seeing the results of his searches.

Summary

In short, what we have really done is:

1. Abstract the google maps API with a wrapper object
2. Create a single $.Deferred() object
3. Queue up calls on the maps API by wrapping the code inside done()
4. Use the async loading option of google maps api with a callback
5. In the maps callback, call resolve() on the deferred object
6. Make the user happy

Demo

In the following demo, you can start searching on an address even before the map loads. Go ahead and try it. I have deliberately put in a 5 second delay on the call to load the maps API, just for a flavor of 3G connectivity!

Don’t forget to browse the code in your Chrome Inspector. You do use Chrome, don’t you? ;-)

As I blogged about earlier, Octopress is a great framework for writing blog posts and packs in all the features for writing a code-centric blogs. Of course, it goes without saying that the blog also looks awesome as if designed by a true designer. Some of the nicer things about writing posts is that there are rake tasks that do most of the grunt work:

• rake new_post[“Just type the title of the post here in plain English”]
  This will create a new file under source/_posts called 2011-09-29-just-type-the-title-of-the-post-here-in-plain-english.markdown
• rake new_page[about]
  This will create a new page under source/about, called index.markdown
• rake preview
  This sets up a local webserver on http://localhost:4000 and starts monitoring the source folder for any changes. It automatically generates the corresponding HTML/CSS for the Markdown/SASS files respectively.

Speed up

If you have just migrated from a Wordpress blog or have lots of posts under your source/_posts, the rake task that generates the HTML output can take a very long time (several minutes). Obviously if you are just working on one post, there is no need to wait for the entire site to generate. What you are looking for is the rake isolate[partial_post_name] task.

Using rake isolate, you can “isolate” only that post you are working on and move all the others to the source/_stash folder. The partial_post_name parameter is just some words in the file name for the post. For example, if I want to isolate the post from the earlier example, I would use

1

rake isolate[plain-english]

This will move all the other posts to source/_stash and only keep the 2011-09-29-just-type-the-title-of-the-post-here-in-plain-english.markdown post in source/_posts. You can also do this while you are running rake preview. It will just detect a massive change and only regenerate that one post from then on.

All set to publish

When you are ready to publish your site, just run rake integrate and it will pull all the posts from source/_stash and put them under source/_posts. Now you can run rake generate and then rake deploy to publish your updated blog.

If these seem like lot of commands to remember, don’t worry, they will become second nature once you do it few times. As a summary, below are all the tasks that we talked about in this post. The description of each task comes from the Rakefile used by Octopress. I just did a rake list -T to get a dump of all the tasks.

• rake new_post[title]: Begin a new post in source/_posts
• rake new_page[filename]: Create a new page in source/(filename)/index.markdown
• rake generate: Generate jekyll site
• rake deploy: Default deploy task
• rake preview: Preview the site in a web browser
• rake isolate[filename]: Move all other posts than the one currently being worked on to a temporary stash location (stash) so regenerating the site happens much quicker
• rake integrate: Move all stashed posts back into the posts directory, ready for site generation

I have been using Wordpress for few years now and have been very happy with its features. In the past year, I have tried several times to change the theme on my blog and also semantify my posts by using Markdown as my de facto style. Of course, none of it happened and I was still using a combination of HTML and Rich Text Editor for formatting my posts.The more I delayed, the more I realized that there were a lot more reasons to NOT like Wordpress:

• I wanted to use Markdown to write all my posts and Wordpress forced me to use HTML. I could certainly use some plugins to upload a markdown file which would then convert it into html, but that meant I had to store these markdown files in the wordpress database: less than optimal.
• Code formatting was not an easy task. I used Live Writer as my primary blog editor and it had a few plugins that can give you inline code highlighting. Although you get real time view of your syntax highlighted code, it internally converted everything to HTML and discarded the original code snippet. Also you had to be careful about editing around that code snippet as a simple delete in the wrong place would require redoing the whole process. I felt it was too much work just to get some code highlighting.
• The Backup and local testing scenario was involving. For backup, I could either export all my posts in WXR format or take a dump of my database. To re-create my blog locally meant getting an installation of MAMP and then importing the WXR or the database backup. I would have preferred a less intrusive approach to try out my wordpress site locally.
• The wordpress technology stack was not very exciting for me. I never really enjoyed PHP and learnt it only to maintain my Wordpress site.

Exploring beyond Wordpress

I had seen a few bloggers use GitHub as their blogging engine with the Jekyll framework to auto-generate their HTML pages from their markdown posts. This was very inviting except for the fact that I had to store all my posts publicly on Github. Even if I purchased a private plan from Github, the storage allocated was quite minimal. GitHub for me was definitely not cost effective.

About this time, I saw a tweet from Matt Gemmell where he migrated from Wordpress to a different engine called Octopress. After reading his blog entry, I realized this was exactly the kind of framework I wanted. Matt has lot more content than I do and seeing him convert his blog successfully gave me the courage to do the same. Thus began an almost 10 day journey to convert my Wordpress blog to an Octopress blog!

There are many things to like about Octopress:

• Write all my posts in Markdown
• Default theme is very beautiful with rich support for styling via Compass/SASS
• Modifying the theme is simple as its based on Jekyll. If you haven’t explored Jekyll yet, I strongly encourage to give it a try.
• Writing plugins is also quite simple and uses the Liquid templating system
• My entire blog is contained within a folder from which I can generate the HTML
• Uploading is taken care with a rake task to deploy (Did I mention Octopress uses Ruby!)
• I can preview my site locally with a simple rake preview command that starts up a local web server. It monitors changes to my blog and auto generates the html. This is great for composing posts and testing on the fly.
• Excellent integration with social features like Google+, Twitter, Disqus, etc.
• The Octopress tagline says its “the blogging framework for hackers” :-)

Migrating Wordpress posts

This was the most elaborate part of the process. Octopress requires that you do write all your posts in Markdown or Textile, however my Wordpress posts were all plain html. So I needed some converter that would do this transformation for me. Luckily on Matt’s blog I read about the exitWP plugin that takes care of this conversion. Although not seamless, exitWP did give me a good starting ground since it converts all the posts to a Jekyll-compliant site.

I did have to go in and change several of my posts that used code snippets. I had been using a variety of code prettifiers over the years and the corresponding HTML was not the best for a Markdown conversion. It did mess up lot of my posts and I spent several hours touching up the Markdown text.

I also got the chance to fix some of my old Urls that were still pointing to my old blog on Live Spaces. I also decided to make all my internal blog links relative and this required a combination of grep/awk and some manual intervention to fix up all the links. Overall it was a fun exercise experimenting with some bash shell commands and a mix of some ruby scripting.

Migrating Wordpress comments to Disqus

Octopress has excellent integration with Disqus, a hosted comment management system. Disqus works by linking all the comments to a specific Url. As long as your posts maintain the same Url you can just use Disqus to import all of your comments into your octopress blog. In my case, my comments were all on Wordpress and I had to first import them into Disqus. As it turns out, this wasn’t a straightforward process.

I started by exporting my comments from Wordpress in the standard WXR Xml format. When I tried to import this file into Disqus, it choked by complaining that the <link> tags were missing. The <link> tag contains the url that links the post to the comments. To fix that I wrote some simple ruby code to update the WXR with the proper <link> tags. Now trying the import again inside Disqus went through without issues and all my comment threads got pulled in. The threads however were using the raw wordpress url (http://blog.pixelingene.com/?p=123) and I wanted to use a more semantic url of the form http://blog.pixelingene.com/year/month/the-post-slug. To fix this I created a simple Url map (CSV) and used the Disqus Url Mapping Tool to fix these links.

Finally with all that done, my comments were safe and sound inside Disqus, with the right permalink-Urls. Now the next part was to link them up with my blog. Luckily this is as simple as specifying a disqus_short_name in the Octopress config file!

Url Rewrites and other changes

Now that I had chosen to use a semantic permalink to my posts, I also had to make sure my existing links to the posts continued working. This was a matter to having some redirects set up on my website. I used the standard Apache directives (RewriteCond, RewriteRule) in my .htaccess to permanently redirect all of my old urls.

A few other things I had to do include:

• 404 page
• Plugins (Liquid Tags) to embed Silverlight apps and Youtube videos
• Change the feed Url from the default /atom.xml to my FeedBurner url

The one thing I havent’ done yet is modify the theme from the default. I’ll probably get to it one of these days.

Epilogue

So that’s my experience with the Wordpress to Octopress migration. Although not a smooth transition, it wasn’t terribly bad and I actually enjoyed the process using a variety of tools. I have tried my best to make sure that all existing wordpress links, images, download links, demos, etc. continue working, but there is always that infinitesimal probability of missing something. If something does break, I’ll find out in one way or other. Until then “enjoy the new blog!”

In JavaScript, if you set a property on the prototype, it is like a static property that is shared by all instances of the Function. This is common knowledge in JavaScript and quite visible in the code. However if you are writing all your code in CoffeeScript, this fact gets hidden away by the way you declare properties.

1
2
3
4
5

class Site extends Backbone.view
  staticProp: "hello"

  initialize: ->
    @instanceProp: "instance hello"

If you declare properties without the @ symbol, you are effectively creating properties on the prototype of the class. This of course works great if you want a shared property but certainly not the way to go if you want per-instance properties. I missed out using the @ symbol and my app went bonkers. This simple oversight cost me fair bit of time debugging it. The right thing to do was using the @property syntax, since I needed per-instance properties. In the code snippet shown above, the staticProp is a property on the prototype of the Site function. @instanceProp is an instance property that will be available on each instance of Site. CoffeeScript translates the above source to the following JavaScript:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

var Site;
var __hasProp = Object.prototype.hasOwnProperty, __extends = function(child, parent) {
  for (var key in parent) { if (__hasProp.call(parent, key)) child[key] = parent[key]; }
  function ctor() { this.constructor = child; }
  ctor.prototype = parent.prototype;
  child.prototype = new ctor;
  child.__super__ = parent.prototype;
  return child;
};
Site = (function() {
  __extends(Site, Backbone.view);
  function Site() {
    Site.__super__.constructor.apply(this, arguments);
  }
  Site.prototype.staticProp = "hello";
  Site.prototype.initialize = function() {
    return {
      this.instanceProp: "instance hello"
    };
  };
  return Site;
})();

As you can see, staticProp (line# 15)is set on the Site.prototype and instanceProp (line# 18) is on the instance (this) of Site. So the take away is:

Be careful while declaring properties in CoffeeScript

CoffeeScript offers a lot of syntax sugar which makes programming JavaScript a lot more fun. Do check out the website for other interesting features.

In the previous post we saw how the D3.js library could be used to render tree diagrams. If you haven’t read that post yet, I would encourage reading it as we will be expanding on it in this post.

Now that we have a nice tree diagram of a hierarchy, it would be good to build in some interactions. After all, given a tree, one would certainly like to click on the nodes and see something happening. Lets do that by animating the path from the clicked node all the way to the root. To pull this off, we need three things:

1. A set of all nodes from the clicked node to the root
2. A set of all links (paths) from the clicked node back to the root
3. A revealing animation that progressively draws the paths from the clicked node to the root

The first two parts are easy and we have already seen that in the previous post. d3.layout.tree has a method called nodes() that gives us the collection of the nodes with their (x, y) locations and links() gives all links between the nodes. We will use the nodes collection to traverse the parent chain from the clicked node all the way to the root. Once we have the nodes along the parent chain, we use that to select the links that lie between each of these nodes. With the links between the nodes, the last part is to animate them into view. The animation part is the most interesting and also the topic of this post.

Animating the path to root

Now that we have the links between the nodes, the next step is to simply draw the SVG Path segments connecting these nodes. We will keep these path segments in its own SVG Group for easier manipulation. To animate these paths, there are two different approaches we can take:

• We can draw the path progressively by setting the SVG Path data in each time interval of the animation OR
• We can use a clipping path for the SVG Group and grow the bounds of the path from right to left. As the clipping path grows in size, it reveals more of the underlying group (which contains the paths), thus animating the path from the clicked node to root. This technique can even be used for progressively revealing complicated graphics.

I am adopting the later approach for this post. To set this up, we first need to add a <clipPath> node to the <svg>. We will also give it an id, so we can reference it later. This clip-path is going to be a <rect> since that is all we need. The code below sets up the clipping path for the <group> that contains the paths to root. The variable animGroupholds the reference to this group.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

var svgRoot = d3.select(containerName)
    .append("svg:svg").attr("width", size.width).attr("height", size.height);

// Add the clipping path
svgRoot.append("svg:clipPath").attr("id", "clipper")
    .append("svg:rect")
    .attr('id', 'clip-rect');

var layoutRoot = svgRoot
    .append("svg:g")
    .attr("class", "container")
    .attr("transform", "translate(" + maxLabelLength + ",0)");

// set the clipping path
var animGroup = layoutRoot.append("svg:g")
    .attr("clip-path", "url(#clipper)");

Note that we have also set up an “id” for the <rect> (#clip-rect), which we will use later for animating the clipping path. The “click” handler To kick off the animation, we will detect a click happening on a node and then draw the path from the clicked node to the root. In the code below we do this using d3.js’ event API: the on(“event”) method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

function setupMouseEvents()
{
    ui.nodeGroup.on('mouseover', function(d, i)
    {
        d3.select(this).select("circle").classed("hover", true);
    })
        .on('mouseout', function(d, i)
        {
            d3.select(this).select("circle").classed("hover", false);
        })
        .on('click', function(nd, i)
        {
            // Walk parent chain
            var ancestors = [];
            var parent = nd;
            while (!_.isUndefined(parent)) {
                ancestors.push(parent);
                parent = parent.parent;
            }

            // Get the matched links
            var matchedLinks = [];
            ui.linkGroup.selectAll('path.link')
                .filter(function(d, i)
                {
                    return _.any(ancestors, function(p)
                    {
                        return p === d.target;
                    });
                })
                .each(function(d)
                {
                    matchedLinks.push(d);
                });

            animateParentChain(matchedLinks);
        });
}

In the the “click” event handler, we traverse the parent links back to the root and record all the ancestors in the path. We then filter all the rendered links (<path>) by picking only the ones that contain the nodes in the ancestors list. This becomes our list of matched links. Using these links we can now finally handle the animation that progressively reveals the emboldened path back to the root.

Back to my Mac root

Since we have all the links that take us from the clicked node to the root, we apply the same method that we used earlier to render the links. Using d3.diagonal(), we can generate the <path> data between the ancestor nodes. Only this time, the path will be rendered with a class set to “selected”, which emboldens the route back to the root.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

function animateParentChain(links)
{
    var linkRenderer = d3.svg.diagonal()
        .projection(function(d)
        {
            return [d.y, d.x];
        });

    // Links
    ui.animGroup.selectAll("path.selected")
        .data([])
        .exit().remove();

    ui.animGroup
        .selectAll("path.selected")
        .data(links)
        .enter().append("svg:path")
        .attr("class", "selected")
        .attr("d", linkRenderer);

    // Animate the clipping path
    var overlayBox = ui.svgRoot.node().getBBox();

    ui.svgRoot.select("#clip-rect")
        .attr("x", overlayBox.x + overlayBox.width)
        .attr("y", overlayBox.y)
        .attr("width", 0)
        .attr("height", overlayBox.height)
        .transition().duration(500)
        .attr("x", overlayBox.x)
        .attr("width", overlayBox.width);
}
  

Once the paths have been rendered, we set the clipping path starting with a zero-width <rect>, which is then animated to the full width of the tree-diagram. It is in this animation of the clipping-path where you will see the path drawing itself out, giving us the final effect we were looking for!

Demo

You can see this code in action here.

In the past few weeks, I have spent some time evaluating some visualization frameworks in Javascript. The most prominents ones include: Javascript InfoVis Tookit, D3 and Protovis. Each of them is feature rich and provides a varieties of configurable layouts. In particular I was impressed with D3 as it gives a nice balance of features and flexibility and allowed me to build just the visualizations I wanted. In this post, I want to take a quick dive into using D3 for building a tree diagram.

A brief course on D3

One of the most striking features of the D3 framework is the use of selections, which allows you to add, update and remove elements in one single chained call. It does this by maintaining an inner-join between the data and the render-elements. This is a pretty powerful feature and greatly aids in building visualizations.

Every render element has an __data__ property that contains the bound data. New render elements will not have the __data__ property and will be created. Existing render elements will get updated and render-elements that have __data__ but no corresponding presence in the new data set will be removed. With that context, the call chain below should look more meaningful.

1
2
3
4
5
6
7

layoutRoot.selectAll("path.link")
.data(links)
.enter()
.append("svg:path")
.attr("class", "link")
.attr("d", link);
  

This selects all SVG Path elements with a class name of ”link” from the parent element: layoutRoot. We enter the selection with a binding to data(), followed by enter(). By entering the selection, we create the inner-join which will automatically know which elements to create and which ones to update. These calls are part of the Selection API. D3 also has APIs for doing animations (Transitions), data processing, computing a varieties of layouts and generating shapes using SVG.

Other powerful features of D3 include:

• Easy mapping of data to Html elements and vice versa
• Flexibility to choose your own render elements (be it html or svg)
• Simple API to configure the layouts and styling
• Animation support with the transition API

Building the Tree

For this blog post, I am going to use sample tree data that looks like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

var treeData = {
    name: "/",
    contents: [
        {
            name: "Applications",
            contents: [
                { name: "Mail.app" },
                { name: "iPhoto.app" },
                { name: "Keynote.app" },
                { name: "iTunes.app" },
                { name: "XCode.app" },
                { name: "Numbers.app" },
                { name: "Pages.app" }
            ]
        },
        {
            name: "System",
            contents: []
        },
        {
            name: "Library",
            contents: [
                {
                    name: "Application Support",
                    contents: [
                        { name: "Adobe" },
                        { name: "Apple" },
                        { name: "Google" },
                        { name: "Microsoft" }
                    ]
                },
                {
                    name: "Languages",
                    contents: [
                        { name: "Ruby" },
                        { name: "Python" },
                        { name: "Javascript" },
                        { name: "C#" }
                    ]
                },
                {
                    name: "Developer",
                    contents: [
                        { name: "4.2" },
                        { name: "4.3" },
                        { name: "5.0" },
                        { name: "Documentation" }
                    ]
                }
            ]
        },
        {
            name: "opt",
            contents: []
        },
        {
            name: "Users",
            contents: [
                { name: "pavanpodila" },
                { name: "admin" },
                { name: "test-user" }
            ]
        }
    ]
};
  

d3.layout.tree() is the starting point for tree layouts in D3. The call to this function returns an object that contains a bunch of methods to configure the layout and also provides methods to compute the layout.

1
2
3
4
5
6
7
8
9
10
11

var tree = d3.layout.tree()
    .sort(null)
    .size([size.height, size.width - maxLabelLength*options.fontSize])
    .children(function(d)
    {
        return (!d.contents || d.contents.length === 0) ? null : d.contents;
    });

var nodes = tree.nodes(treeData);
var links = tree.links(nodes);
  

You can ignore the call to the size()method, it is just setting the width and height of the tree, passing in as a 2-element array ([w,h]). If you are curious why I have the size.height as my first element, its because I am rotating the tree by 90 degrees. We will see more about his further down the post. The children()function is more interesting, as it tells the tree layout about the child nodes of the data-item. In my example, the contents property represents the children, as seen in the treeData variable above.

The next two methods, nodes() and links()does what you expect. It recurses through the data (by calling children() on each item) and computes the layout info for each data item. The links() method takes the output of nodes() and computes the edges between the different nodes. We are storing this computed layout in the nodes and links variables, each of which is a flat array of elements. Each element in the nodes array is a wrapper around the original data item with augmented layout info, whereas each element in the links array is a wrapper around the node item. Below you can see these wrapper objects:

The node item is augmented with the children, parent, x and y properties. The link item just has the source and target properties, which represents the edge from the parent to the child respectively. You can see that the source and target properties point to the node objects.

Rendering the tree

Now that we have the layout computed, the next step is to setup the UI elements that render the data and layout. For the tree diagram we are going to use SVG, although the same layout can also be achieved using plain Html,CSS with absolute positioning. To create our UI elements, we will use D3’s selection API, as mentioned above, to generate the SVG elements and set properties based on the computed layout.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

/*
     <svg>
         <g class="container" />
     </svg>
  */
 var layoutRoot = d3.select(containerName)
     .append("svg:svg").attr("width", size.width).attr("height", size.height)
     .append("svg:g")
     .attr("class", "container")
     .attr("transform", "translate(" + maxLabelLength + ",0)");


 // Edges between nodes as a <path class="link" />
 var link = d3.svg.diagonal()
     .projection(function(d)
     {
         return [d.y, d.x];
     });

 layoutRoot.selectAll("path.link")
     .data(links)
     .enter()
     .append("svg:path")
     .attr("class", "link")
     .attr("d", link);


 /*
     Nodes as
     <g class="node">
         <circle class="node-dot" />
         <text />
     </g>
  */
 var nodeGroup = layoutRoot.selectAll("g.node")
     .data(nodes)
     .enter()
     .append("svg:g")
     .attr("class", "node")
     .attr("transform", function(d)
     {
         return "translate(" + d.y + "," + d.x + ")";
     });

 nodeGroup.append("svg:circle")
     .attr("class", "node-dot")
     .attr("r", options.nodeRadius);

 nodeGroup.append("svg:text")
     .attr("text-anchor", function(d)
     {
         return d.children ? "end" : "start";
     })
     .attr("dx", function(d)
     {
         var gap = 2 * options.nodeRadius;
         return d.children ? -gap : gap;
     })
     .attr("dy", 3)
     .text(function(d)
     {
         return d.name;
     });
  

The code above creates the root container (SVG with a Group (<g>) element) and adds the edges first followed by the nodes and text labels. This order ensures that the nodes and labels always stay on top of the edges. You can also see the use of the .data().enter() method chain to setup the inner-join for binding data to UI. The attr() method is used to set properties on the SVG elements. One thing to note here is that we are creating a group (<g>) element to hold both the node (<circle>) and the label (<text>). Also this group element is translated by [y, x], ie. by interchanging the original layout co-ordinates. We do this to rotate the tree by 90 degrees. This rotation makes the labels more readable.

I would heartily recommend D3.js to anyone looking into building 2D visualizations in Javascript. Although it has a slight learning curve, it is worth the effort and allows building some fancy visualizations. Do take a look at the examples to see the capabilities of this library.

Demo

You can take a look at the tree demo here.

Of late, I have been building some Html/Javascript apps and exploring a bunch of javascript libraries, including the usual suspects (jQuery, jQuery UI, jQuery template, underscore, etc). The more interesting ones are visualization libraries like d3, isotope, highcharts. In this post, I will focus on a specific scenario in the isotope.js library.

Isotope.js

Isotope.js is a neat jQuery plugin for doing layout animations and provides a bunch of cool layout modes. When it is applied to a list of items, it lays them out in one of the several layout-modes and animates their positions upon changes in the list. It also provides sorting and filtering hooks which also trigger layout animations when applied. The GIF animation below gives a glimpse of the ‘masonry’ layout mode.

The way isotope treats filters is by looking for a particular CSS class on each of the items. If the item has that class, it is included in the filtered list. This form of filtering is very declarative and easy to apply.

However if you need more complex filtering, you will have to do some leg work. You will have observed in other toolkits, frameworks that filter is generally considered as a lambda-function that will be invoked for each item in the list. This is a convenient form of providing the filtering logic.

Isotope, however does not think that way and only looks at the class attribute of the item and matches on the supplied filter-class-name. This means, we will have to pre-process the items, apply the complex filtering logic and attach an ad-hoc class to the item. That is the key to complex filtering in isotope: a pre-processing step that attaches an ad-hoc class on matched items, which is then used by isotope.

The code-snipped below shows the pre-processing step before invoking the $.isotope() function. Look for the comments starting with [1], [2], [3].

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

function createTestData()
{
    var people = _.range(0, 100).map(function(p)
    {
        return {
            name: "Name-" + p,
            age: Math.round(Math.random() * 50),
            email: "email-" + p + "@company.com",
            image: "css/ninja/" + Math.ceil(Math.random() * 6) + ".png"
        }
    });

    $("#person-template").tmpl(people).appendTo("#people-list");
    $("#people-list").isotope({
        layoutMode:'masonry'
    });
}

$(document).ready(function()
{
    createTestData();

    // [1] Filtering function invoked for each item
    function predicate(data) {
        return data.age > 25;
    }

    // [2] Applies the ad-hoc 'matched-item' class to items that match the filter predicate
    $("#invoke").click(function(){
        $(".isotope-item").each(function(){
            var item = $(this).tmplItem().data;

            // [3] Apply the 'matched-item' class
            if (predicate(item)) $(this).addClass("matched-item");
            else $(this).removeClass("matched-item");
        });
        $("#people-list").isotope({
            filter: '.matched-item'
        });
    });

    $("#reset").click(function(){
        $(".isotope-item").each(function(){

            // Clear the class
            $(this).removeClass("matched-item");
        });

        $("#people-list").isotope({
            filter: '*'
        });
    });
});
  

So filtering in isotope can still be done the traditional way, just that we need a pre-processing step.

Demo

For a quick demo of the isotope-filtering, click here.

A few days back while I was busy designing some UI for a Silverlight app, I accidentally hit upon this fun hack.

If you assign a shared Brush resource to the CaretBrush property of the TextBox control, then you start seeing some crazy blinking-light effects at places where the shared Brush is used. It is really fun the first time you see it happening and then on, it is sure to cause some Dilbert style “Mahjobbis Crappus”.

Fortunately the solution is quite simple: DO NOT use a shared brush resource for the CaretBrush !

Demo / Download

Below you will find a sample Silverlight app that shows this in action:

Download Solution/Project

After having worked full-time for several years in the Corporate world, I have decided to make a career change and jump on to Consulting. I have joined my friends at
, where I’ll be working in the Financial district of New York, building solutions using Microsoft .Net, C\#, WPF, Silverlight, Html5 + JavaScript and others.

I have known some people at “the Lab” for several years, mostly when they were consulting with my earlier company. So it feels nice to go into a familiar crowd. Lab49 works on a variety of technologies and have a rich portfolio of clients. Many of the folks working here are experts on some technologies and also have active blogs. It is surely a great place to learn new things and work on exciting projects.

I think it will be fun.