IPEP 5: Notebook JavaScript organization - ShuaiYAN/ipython GitHub Wiki

Author:	Bussonnier Matthias <[email protected]>
Author:	Brian Granger <[email protected]>
Status:	Draft
Created:	2012-10-23
Updated:	2012-10-23

The Notebook's JavaScript implementation is growing more and more complex. This is making it difficult to scale its development and write good clean code. Some specific problems we are having:

• Lots of JS files with complex, implicit dependencies.
• Painful management of all of those files in our HTML templates.
• All JS files are in a single directory. As we add more pages and JS files, this becomes unmanageable.
• No minimization/build logic for deployment.
• Difficult to grab some of the JS and reuse it in other projects.
• Model and view logic is welded together.

This is a proposal to explore what we might do about these problems.

CoffeeScript (http://coffeescript.org/) is a mini-language that sits on top of JavaScript and compiles into it. It is praised for having a more compact syntax than JavaScript and fixing a lot of the design issues of JavaScript.

• Much more compact syntax than JavaScript.
• Depending on who you ask, it may or may not be more readable.
• List comprehensions, iterables, default function arguments.
• A nice Pythonic class model with inheritance.
• Can be compiled to JavaScript on the server or optionally be loaded into the browser as text/coffescript and interpreted on the fly.
• Can be used side-by-side with JavaScript so our transition could be gradual.

• Syntax is almost a linear combination of JavaScript and Python, making it a bit difficult to keep things straight.
• Either we have to translate in the browser (slow) or add a bulid step to our dev process.
• Because CoffeeScript compiles down to JavaScript, debugging requires a developer to muck through generated JavaScript. This means developers will have to be very good at JavaScript. This means CS is not a simple replacement for JS in the dev brain.
• It adds complexity with a questionable benefit.

Recommendation: Don't use CoffeeScript yet, but continue to learn more and discuss it.

Backbone (http://backbonejs.org/) is a very popular MVC library for JavaScript. It allows you to create JS models and synchronize them with DOM based views and backend server storage. It might help us to write cleaner JS code, especially in separating our model data from our view logic. It looks very nice, but its notion of a model is very limited and flat. For example, our data model for the notebook is very nested and Backbone doesn't have a way to build models that contain models that contain models. It only has models with attributes and those models can be organized into collections. However, this design probably reflects the reality of talking to restful web services and we would probably be fine. Using Backbone would basically amount to a complete rewrite of the JS code for the notebook. This would also involve rewriting the notebook web service.

Recommendation: It is too much to undertake at this point. But let's continue to learn more about Backbone and clean up our JavaScript code to prepare for a possible future transition.

Currently our JS code is a bunch of files that we include by hand on each page. We keep these files in a single directory and don't have the ability to track dependencies. This makes it difficult to manage and the situation is getting worse as the code base grows. It would be like developing in Python without a package/module structure. Painful.

CommonJS (http://www.commonjs.org/) is an API created to add package management and importing to JavaScript. It uses the form:

mymod = require('mymodule')

which is the equivalent of:

import mymodule as mymod

in Python. The CommonJS API is synchronous and ignores the realities of browsers. Because of this it is primarily used in server side JS code (Node.js).

RequireJS (http://requirejs.org/) is a JavaScript library that attempts to bring the features of CommonJs into the asynchronous context of browsers. It is gaining lots of traction, but is no small dependency - if you buy into it, you have to completely reorganize your JS code.

Advantages:

• Would allow us to organize our JS code into clean modules and subdirs.
• Modules could explicitly declare which modules they depend on.
• Would greatly simplify our HTML templates - we would only need to include 1 or 2 JS files.
• Works with browser and server side code.
• Has a tool for producing production/deployment minimized code.

Disadvantages:

• It is not simple at all. Everything is based on callbacks.
• Anyone who wants to use our JS code would have to buy into using RequireJS. This is huge...
• It is a little painful to integrate Js libraries (jQuery, Bootstrap, etc.) that don't use RequireJS.

Recommendation: We should build a sample project that uses RequireJS to explore its features. This sample project should include jQuery, Bootstrap and organize JS code in a way similar to what we would need. We should use this sample project as a test to see how we should organize our JS code.

Some points to consider during this reorganization:

• We probably want to organize our code into different subdirs, maybe by which page it goes with.
• Keep in mind that we may end up rewriting our JS code using CoffeeScript and/or Backbone in the future. We need a structure that is flexible enough for that.
• We want it to be easy for people to reuse our JavaScript.

Bower (http://twitter.github.com/bower/) is a simple JS+CSS package management tool that is written in node.js. Packages are simply git repos with a special component.json file. This file is optional, but can be used to declare dependencies. By default bower installs in the component subdirectory of cwd. So the following commands:

bower install bootstrap
bower install codemirror

would result in the directories:

/components/jquery
/components/bootstrap
/components/codemirror

Notice that I didn't have to install jQuery - Bootstrap declares jQuery as a dependency and bower automatically finds and installs it. Bower knows about versions of packages by looking at the tags on github. It does have some constraints on the format of version strings it can handle (it fails on tags like "v1.0").

Recommendation: I think we should embrace the bower vision of JS+CSS assets and use it in a number of ways:

• We should organize our static dir in a manner that matches bower's approach. This would mean putting all of the IPython css+js under static/ipython and putting a component.json file in that directory to declare our dependencies.
• We should start to replace our external dependencies with versions that can be installed using bower. This would include jQuery, Bootstrap, jQueryUI, CodeMirror, etc. If we need to change their code, we should do so by monkey patching so we can always use the originals. This will greatly improve the situation for packagers as the packages come with a component.json file that has all of the information about where the package came from (git repo, branch, head, version).
• We should adopt bower as the tool for installing JS plugins for the notebook. This means that each JS plugin should simply be a git repo with a component.json file. This will enable plugins to depend on other plugins and installation will be simple. Because JS plugins can also include Python code, they optionally include a top level setup.py file to install included Python packages/modules. It is important that this script is top-level so things like easy_install can find it.

There are a couple of different approaches we could take for organizing our files.

In this model, the top level directories are associated with a single page:

page1/
page1/js
page1/less

page2/
page2/js
page2/less

The problem with this approach is that there are assets we use on multiple pages. We could start to separate these out like this:

base/
base/js
base/less

kernel/
kernel/js

But you can probably see where this is heading...

In this model, the top level directories are associated with a small unit, something like a module or set of modules:

/cell
/cell/js
/cell/less

/notebooklist
/notebooklist/js
/notebooklist/less

/toolbar
/toolbar/js
/toolbar/less

This too has problems. How many classes/modules do we put into each directory. Is it one class per directory? A set of classes? If we move to an MVC based approach, where do we put our models and view? In each subdir?

In this approach, were would have separate directories for models, views and controllers:

/models
/models/toolbar.js
/models/cell.js

/views
/views/toolbarview.js
/views/cellview.js

Here, I am not quite sure where to put our css files...

There are two other types of static assets we have to handle:

1. HTML templates. These come in two flavors: a) Jinja2 templates that our server needs to render and b) JS rendered templates handled by something like Mustache or Underscore.
2. Main scripts for each page. Each page is going to have a script where requirejs is configured, require is called and that pages objects are instantiated.