Edit: Some of the material in the Assembling a Frontend Stack posts is outdated. Specifically, I no longer use Bower in any of my projects.

How to organize your Javascript code

Even a marginally sophisticated webapp will have two bundles of Javascript:

• 3rd party code like jQuery or Ember
• your code; the code that you write that describes your app

In 2014, these two kinds of code are going to have to live separately. Ideally, a single build tool could fetch 3rd party dependencies and build them alongside your code. But the current Javascript landscape just isn't there yet. 3rd party frontend library developers don't all agree on the "right" way to package and distribute their code.

For the time being, the best thing we have is Bower. Most 3rd party frontend packages distribute themselves as a Bower package.

"Why not just download the 3rd party scripts that I need and stick them in my /public/js/vendors/ directory?" you ask. Because that quickly becomes unwieldy. With a managed 3rd party system like Bower, you get:

• A straightforward way to see what version of each of your 3rd party libraries you are using, if there's a new version available, and a clear way to upgrade them.
• A clear distinction between your project's files and 3rd party files. Your project's bower.json configuration lists all of your project's needed 3rd party packages. That bower.json file goes into your revision control system, but the pacakges themselves don't.
• Portability. When you or someone else needs to stand the project up on another server or another workstation, they just type bower install, and all of those 3rd party dependencies are right where you expect them to be.

Getting started with gulp and Bower

Let's set up the tools and basic config files we'll need to assemble our 3rd party dependencies with Bower.

Directory structure

Throughout these posts, we will use the following directory structure:

+-- src
|   +-- js
|   +-- less
|   +-- hbs
|-- public
|   +-- js
|   +-- css

Setting up your environment

• Create a directory for this project and then create the subdirectories described above.
• Install Node if you don't already have it (installing with a package manager is best). Node powers all of the tools we'll be using.
• cd to your project directory
• Globally install bower if you don't already have it:
  npm install -g bower
• Globally install gulp:
  npm install -g gulp
• Set up a package.json file with npm init. This file will keep track of the plugins and libraries we'll use with gulp. The suggested defaults are fine.
• Set up a bower.json file with bower init. This file will keep track of our 3rd party dependencies like Backbone, Bootstrap, and the like. Again, the defaults are fine unless you have reason to change them.

3rd party dependencies

Installation

Our project will use some common 3rd party libraries: Backbone, Handlebars, and Bootstrap.

bower install --save-dev backbone handlebars bootstrap

Don't forget --save-dev. This causes these packages to be written to the devDependencies section of your bower.json file.

Assembling the 3rd party dependencies for our app

We now have our frontend dependencies sitting in subdirectories in the bower_components directory. How should we add these dependencies as script tags in our app?

One option is to copy each one to a public/js/vendor/ directory and include them like
<script type="text/javascript" src="/js/vendor/backbone.js">.

But this would quickly become cumbersome. In addition, each of these files may have to be added to your testing configuration. So this seems less than ideal.

Instead, let's combine all of our 3rd party JS files into a single file to be included on our webapp page. Enter gulp.

Using gulp to assemble 3rd party scripts

Initial gulp setup

• We already installed gulp globally on our system, but now we need to install it for this project.
  npm install --save-dev gulp
• We're going to use a npm package called main-bower-files to get a list of the JS files from our bower packages.
  npm install --save-dev main-bower-files
• We're also going to use the standard gulp plugin for concatenating a list of files: gulp-concat.
  npm install --save-dev gulp-concat

Again, don't forget --save-dev. Using --save-dev writes each of these libraries to the devDependencies section of your package.json file, making your project portable.

Create your gulpfile

Create a new file called gulpfile.js in your project root like this:

"use strict";

var gulp = require('gulp'),
  mbf = require('main-bower-files'),
  concat = require('gulp-concat');
  
gulp.task('bower', function () {
  gulp.src(mbf({includeDev: true}).filter(function (f) { return f.substr(-2) === 'js'; }))
    .pipe(concat('vendor.js'))
    .pipe(gulp.dest('public/js/'));
});

This gulpfile defines a single task called bower that concats the JS files installed with Bower to "vendor.js" and writes that file to the public/js/ directory.

You can see gulp's simple API at work.

• gulp.src() produces a readable stream of files. In our case it gets a list of all the main files specified in our bower packages using main-bower-files (mbf() here), filtered to include only files ending with "js".
  To see the output of the main-bower-files plugin, add a
  console.log(mbf({includeDev: true}));
  statement to the beginning of the function.
• .pipe() moves our files through the concat plugin and concats the results to a file called vendor.js
• gulp.dest produces a through stream; that's useful for writing to a directory (you can also read a through stream, so we could continue to .pipe() if we wanted to). In our case, we write our vendor.js file to public/js/.

Save the gulpfile and run your task:
gulp bower

You should now have a public/js/ directory containing vendor.js.

When you include vendor.js on your page like
<script type="text/javascript" src="js/vendor.js">
you will get access to window variables like

• window.Backbone
• window.$
• window._
• window.Handlebars.

Organizing our project's Javascript: CommonJS Modules and Browserify

With our 3rd party dependencies in place, we're ready to start writing our own Javascript code. But how should we arrange our code?

It is common knowledge that jamming all of your Javascript into a single file is a bad idea. It makes merging a nightmare, it's difficult to locate unused code, and it makes testing much more difficult. Separate files that reference a global namespace is a little better, but you end up with lots of ugly boilerplate like this:

var MyApp = window.MyApp || {};  
MyApp.views = MyApp.views || {};  
MyApp.views.controls = MyApp.views.controls || {};  
MyApp.views.controls.Button = Backbone.View.extend({});

The solution: modules.

CommonJS

CommonJS is the module system used by Node.js. It's simple - dependencies are declared using the require keyword and module.exports is used to declare what the module exports. It works well on the web thanks to substack's Browserify tool.

An aside about AMD

Despite a lot of good ideas in AMD, here are two reasons I prefer this CommonJS/Browserify scheme over AMD/RequireJS:

1. CommonJS/Browserify assembles all your JS into a single file, which is included with a single script tag on your page. With AMD, the browser requests each module separately. That could be scores of requests. Because of browser caching, the single file approach seems better to me. See Tom Dale's post on this subject. This may change when we get HTTP 2.0 and multiplexed requests.

2. The Browserify approach means that unused modules automatically stay out of your app. Only those files containing modules that are actually required by another module are added to the build. That single file you include on your page will always contain the minimum amount of code needed to run your app. Old code stays out of production.

Module form

Let's create a Backbone view for our first module, a simple <table> view that doesn't really do anything. Create
src/js/views/Table.js:

(function () {
  "use strict";
  
  var Backbone = window.Backbone;
  
  module.exports = Backbone.View.extend({
  
    tagName: 'table'
    
  });
  
}());

Notice that our Backbone dependency is declared with var Backbone = window.Backbone. Why declare a reference to a window variable we already have access to? In my opinion, this is good practice because all of your module's dependencies are clearly defined at the top of the file. When the day comes that you want to port this code to some other module system, it will be simple (it could even be automated). You will also avoid adding a bunch of global variable declarations to your code linting configuration.

Now let's create the file that will kick off our app on pageload. Create
src/js/app.js:

(function () {
  "use strict";
  
  var $ = window.$,
  	TableView = require('./views/Table');
  
  $(function () {
  	$('body').append(new TableView().render().el);
  });
  
}());

You can see here that app.js imports our Table view with `TableView = require('./views/Table').

The code from src/js/app.js will kick off our app and it also serves as the root node in the module dependency chain. Pointing Browserify at src/js/app.js will pull all modules necessary for our app into the build.

Setting up the Browserify build in gulp

First install the gulp-browserify plugin:
npm install --save-dev gulp-browserify

Next, add the configuration code to your gulpfile.js:

"use strict";

var gulp = require('gulp'),
  mbf = require('main-bower-files'),
  concat = require('gulp-concat'),
  browserify = require('gulp-browserify');
  
gulp.task('bower', function () {
  gulp.src(mbf({includeDev: true}).filter(function (f) { return f.substr(-2) === 'js'; }))
    .pipe(concat('vendor.js'))
    .pipe(gulp.dest('public/js/'));
});

gulp.task('browserify', function () {
  gulp.src(['src/js/app.js'])
    .pipe(browserify())
    .pipe(gulp.dest('public/js/'));
});

Run the new browserify task:
gulp browserify

You should now have public/js/app.js that contains all of the modules for your app.

An aside about Bower and Browserify

Since we're using CommonJS modules and Browserify for our webapp, why not install 3rd party dependencies using npm and simply import in our modules like var Backbone = require('backbone'); or var Handlebars = require('handlebars');?

This would indeed be ideal. Unfortunately, it is not yet a reasonable approach. Backbone and some other frontend libraries are maintained as npm packages, but many others are not. Bower remains the most consistent source for frontend packages.

You now have a concise pattern for cleanly separating modules in your system while incorporating 3rd party dependencies the way the authors of those dependencies probably envisioned (as global window variables).

Read on to add Handlebars and JSHint to the build.