Scalable JavaScript Application Architecture

My Summary: Nicholas showed a technique to build large scale applications, one which he used during his time as an engineer at Yahoo. The technique had specific relevance to single page applications all though could be employed on Multiple page applications as well. The architecture he described split an application into multiple tiers with each tier having specific roles and responsibilities and loosely coupled so that elements could be removed, replaced or updated. Slides from the talk are over on slide share.

Nicholas Zakas is a regular speaker on the JavaScript circuit but for some reason I’d always missed his talks at other conferences. So when I saw his name on the Full Frontal 2011 speaker list I knew I was in for a treat. His talk absolutely delivered the promise of the abstract, a solid talk that gave practical advice on how to build large JavaScript applications.

imageThe Abstract: Building large web applications with dozens of developers is a difficult task. Organising the engineers around a common goal is one thing, but organising your code so that people can work efficiently is another. Many large applications suffer from growing pains after just a few months in production due to poorly designed JavaScript with unclear upgrade and extension paths. Learn the tips, tricks, and techniques that allowed large sites such as My Yahoo! and the Yahoo! homepage to continue to grow, scale, and change over time without throwing away previous work. This talk isn’t specific to any JavaScript library, rather, it gives you new ways to apply the libraries you’re already using. The principles of good, loosely-coupled design apply to any system, and you’ll learn how this can help your application today.

Nicholas took the stage and asked “How many people are building single page JavaScript applications?” over half the audience raised their hands. Well, he continued, lots of people are working on single page apps but what I’m going to talk about works for multi page apps as well.

Many people are using Application frameworks like backbone.js, but many don’t realise it’s actually pretty easy to build your own. According to Nicholas, frameworks are like a playground, there are a bunch of things happening and the framework ties them all together. He was quick to add that a framework was not a replacement for jQuery or YUI but rather it sits on top of those base libraries.

Everything on a webpage is a Module, explained Nicholas, he then drew an analogy between web development and building the international space station by saying different people from different locations can build small parts and they go to create a much larger thing. He defined a module as:

web application module (n)
1 : an independent unit of functionality that is part of the total structure of a web application

All modules should be able to live on their own and this is achieved using loose coupling. Each module has it’s own sandbox. A sandbox sits on top of the application core. It acts like a gate keeper. Modules know about the sandbox but are unaware of other modules.

In total the Application Architecture consists of:

  • Modules
  • Sandbox
  • Application Core
  • Base Library

Each part of the architecture is like a puzzle piece, no single piece needs to know what the completed puzzle looks like, all that matters is that the piece does its own job correctly. Nicholas then went on to explain each of the sections in turn.

Modules

It’s the modules job to create a meaningful user experience. These modules are made up of JS, HTML and CSS. The web application is what is created when all the modules work together.

Core.register("module-name", function(sandbox){
	return {
		init: function(){
		//constructor
		},
		destroy: function(){
		//destructor
		}
	};
});

Modules are like little kids. They need rules:

  • Hands to yourself
    • Only call your own methods or those on the sandbox
    • Don’t access DOM elements outside of your box
    • Don’t access non-native global objects
  • Ask, don’t take
    • Anything else you need, ask the sandbox
  • Don’t leave your toys around
    • Don’t create global objects
  • Don’t talk to strangers
    • Don’t directly reference other modules

Modules must stay within their sandboxes, this can be uncomfortable as developer just want to use YUI or jQuery directly, but by using modules you can build things quicker.

Sandbox

The sandbox ensures a consistent interface, each module only understand the sandbox. The sandbox acts like a security guard, you may not, for example, want the module to set cookies without going through a sandbox.

Core.register("module-name", function(sandbox){
	return {
		init: function(){
		//not sure if I'm allowed...
			if (sandbox.iCanHazCheezburger()){
				alert("thx u");
			}
		},
		destroy: function(){
		//destructor
		}
	};
});

Sandbox Jobs

    1. It provides consistency
    2. It provides security – Determine which parts of the framework a module can access
    3. It provides communication – Translate module requests into core actions

Nicholas made the point that you should take time to design the sandbox as it’s very difficult to change it later.

Application Core

The application core is the gut of the application,  it controls the application. It will start modules and also will destroy modules. So a module might look something like this:

Core = function () {
    var moduleData = {};
    return {
        register: function (moduleId, creator) {
            moduleData[moduleId] = {
                creator: creator,
                instance: null
            };
        },
        start: function (moduleId) {
            moduleData[moduleId].instance =
			moduleData[moduleId].creator(new Sandbox(this));
            moduleData[moduleId].instance.init();
        },
        stop: function (moduleId) {
            var data = moduleData[moduleId];
            if (data.instance) {
                data.instance.destroy();
                data.instance = null;
            }
        }
    }
} ();

And the sandbox might look something like this:

Core = function(){
    return {
        //more code here...
        startAll: function(){
            for (var moduleId in moduleData){
                if (moduleData.hasOwnProperty(moduleId)){
                    this.start(moduleId);
                };
        }
        },
        stopAll: function(){
            for (var moduleId in moduleData){
                if (moduleData.hasOwnProperty(moduleId)){
                this.stop(moduleId);
                }
            }
        },
    //more code here...
    };
}();

You’d then register the modules like this:

//register modules
Core.register("module1", function(sandbox){ /*...*/ });
Core.register("module2", function(sandbox){ /*...*/ });
Core.register("module3", function(sandbox){ /*...*/ });
Core.register("module4", function(sandbox){ /*...*/ });
//start the application by starting all modules
Core.startAll();

The application core controls the communication between modules. Nicholas used the example of twitter (circa 2009) as a good example of a modularised application, where the different modules need to communicate. When you update your status, the status module needs to let the timeline module know a change has happened.

Nicolas went on to show an example of tight coupling where one module called another module. Explaining that tightly coupled modules were difficult to maintain.

TimelineFilter = {
    changeFilter: function (filter) {
        Timeline.applyFilter(filter);
    }
};
StatusPoster = {
    postStatus: function (status) {
        Timeline.post(status);
    }
};

Creating modules that notified the sandbox that something had changed was a better approach, since the sandbox could then choose if it should do something. Loose coupling ensures your code has longevity, Nicholas stated that the code you write should last 5 years – taking a long term outlook on the code you write ensures you take care of it and that it is extensible.

Nicholas recommended that the application core should handle errors and pointed the audience to another of his presentations Enterprise JavaScript Error handling for further information.

In summary the Applications Core’s Jobs are:

  • Manage module lifecycle
    • Tell modules when to start and stop doing their job
  • Enable inter-module communication
    • Allow loose coupling between modules that are related to one another
  • General error handling
    • Detect, trap, and report errors in the system
  • Be extensible
    • The first three jobs are not enough

According to Nicholas planning for extension is extremely important, since anything built for extension can never be obsolete. You should plan for common extension scenarios such as Error Handling, Ajax Communication, New module capabilities and General Utilities. If you use a sandbox and make all your AJAX communications through it, to a single point in your application core, you can switch the AJAX communication out for another type in just one place. It will then be changed for your entire application.

Base Libraries

According to to Nicholas most applications are too tightly coupled to their base library, he pointed to a presentation by Joseph Smarr called High Performance JavaScript  which suggests you should use libraries like jQuery as a scaffolding; only the application core should know what base library is being used. This approach ensures that you can swap out your base library at anytime.

In general a base library should do the following jobs:

  • Browser normalization
    • Abstract away differences in browsers with common interface
  • General-purpose utilities
    • Parsers/serialises for XML, JSON, etc.
    • Object manipulation
    • DOM manipulation
    • Ajax communication
  • Provide low-level extensibility

If you follow Nicholas’s advice then you will end up with an architecture where:

  • Only the base library knows which browser is being used
  • Only the application core knows which base library is being used
  • Only the sandbox knows which application core is being used.
  • Each module knows nothing except that the sandbox exists.
  • And finally no single part of the web application knows about the web application.

By structuring your application this way Nicolas suggested that you could reuse your framework across multiple applications. Also because each piece had it’s own unique job it could be tested individually.

The real benefit, according to Nicolas, was the ability to build highly scalable web applications where pieces could be removed at will.

The talk provided a great introduction to design patterns, Nicolas’s delivery was clear and well paced, and made for a great talk that left me thinking that I should probably apply the same architectural rigor that I apply on the servers side to my frontend development.

Published by thebeebs

Thebeebs is a Canadian pop singer, songwriter, actor and HTML5 junkie. Throughout his rise to fame, Thebeebs has been nominated and awarded numerous accolades, winning Artist of the Year at the 2010 American Music Awards, and being nominated for Best New Artist and Best Pop Vocal Album at the 53rd Grammy Awards. Thebeebs is considered a teen idol, and has been subject to acclaim from fans, as well as criticism and controversy from matters concerning his popularity and image.

7 Comments So Far, what do you think?

  1. Pingback:The Morning Brew - Chris Alcock » The Morning Brew #981

  2. John Brown

    I am trying to get behind this scalable module programming pattern, and I have watch the Zakas talk and seen the script examples here mimicked across several web pages. I am continually confused by the Sandbox element, though, and the problem is the same on each page.

    You call:
    new Sandbox(this)
    in Core.start() and pass that . Yet there is no definition for Sandbox. And immediately after, you call:
    moduleData[moduleId].instance.init();
    which has the Core directly call the module, bypassing the sandbox object it was passed.

    Also, how does the module notify the sandbox that then notifies Core, and how does the core pass along that info to all the modules and check if they are listening for that event?

    In a small example I created in trying to answer these questions, I made a simple Sandbox object:

    var Sandbox = function(core) {
    var core = core;
    var strPrivate = ‘Private’;

    return {
    notify: function(opt){
    core.notifyAll(opt)
    },
    listen: function(moduleID) {
    //what goes here?
    }
    }
    };

    then added to Core:

    notifyAll: function (note) {
    for (var moduleId in moduleData){
    if (moduleData.hasOwnProperty(moduleId)){
    try {
    moduleData[moduleId].instance.handleNotification(note);
    } catch(err) {
    // errors
    }
    }
    }
    },

    Again, though, this bypassed the Sandbox and hits the module completely.

    Sorry if this is rambly. I have been trying to get a better view of the whole framework, and the Sandbox and passing events has me stumped.

  3. flosse

    @thebeebs Thanks for that summary of Nicholas talk!

    @John

    > You call: new Sandbox(this) in Core.start() and pass that .
    > Yet there is no definition for Sandbox.
    Of course the Sanbox function has to be defined globally before ;-)

    > And immediately after, you call:
    > moduleData[moduleId].instance.init();
    > which has the Core directly call the module, bypassing the sandbox object it
    > was passed.

    You’re right. You could so something like sanboxes[moduleId].init(); whereas the sandbox calls the init method of the module. So the question is what is the difference? I think it would be more clearly and you would be able to implement different sanboxes. Thanks for that thought!

    > Also, how does the module notify the sandbox that then notifies Core, and how
    > does the core pass along that info to all the modules and check if they are
    > listening for that event?

    Your have to hold the subscriptions of all modules in the core. Then you’re able
    to notify the modules that are interested in a special event.

    I created a little framework that implements some ideas of Nicholas presentation.
    You could have a look at https://github.com/flosse/scaleApp or http://scaleapp.org/
    It’s written in Coffeescript but the ideas are the same.
    Maybe it helps you? I’d love to get some feedback!

  4. Steve Serps

    This an area that really needs explaining to me again and again, I will get it but I know for fact its never as simple as building a playground. There are certain part of the app I just can’t figure out on my own! Good summary of what must of been quite an interesting conference

  5. Pingback:CSS-selector-based templating – example with JavaScript » GreWeb

  6. Aran Mulholland

    A fairly simple implementation that follows this talk fairly closely can be found here: https://github.com/aranm/scalable-javascript-architecture

  7. web content writing services

    It’s difficult to find well-informed people for this subject,
    but you seem like you know what you’re talking about!

    Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>