Blacklight is a Content Management System (CMS) and web server that runs on the Node.js platform and works with Apache Sling-based content repositories for data storage.

Blacklight uses the Handlebars templating language to generate HTML. Handlebars is intentionally limited in its capabilities. When more complicated data processing or business logic is required, Blacklight does that work using dedicated model processors, which are written in Node.js. As a developer you're encouraged to organize all the Blacklight code needed to customize your site around modules and components. These two concepts are described below.

Modules

There are three categories of module to consider when using Blacklight:

1. Third-party NPM modules -- As a regular Node.js app, Blacklight uses NPM to manage third-party libraries

2. Internal Node.js modules -- Besides external third-party NPM modules, Blacklight also makes extensive internal use of Node's preferred CommonJS format. Much of Blacklight's behaviour is cusomtized and controlled by placing CommonJS module files in particular places, with particular file names, and which export particular properties. For example, every Blacklight model processor is implemented as a CommonJS module stored in a predictable place.

3. Blacklight site modules -- For the purposes of this document, this is the most important type of module in Blacklight to understand. Most websites have distinct areas, each with concerns that are related but separate from the other site areas. Blacklight enables the organization of code into modules divided along these same lines of concern by isolating that code into "Blacklight site modules." (Sometimes also simply referred to as "Blacklight modules.") This separation helps ensure each distinct area of concern will remain isolated in its own code base, and has advantages that include simplifying testing, avoiding unintentional side effects, improving the ability to reason about the code, and minimizing the build time of each site module.

Note, Blacklight site modules are not in and of themselves NPM/CommonJS modules. Instead, each Blacklight site module is simply a collection of related code, organized a particular way, and stored in its own Git repository. Blacklight site modules can be created, installed, and managed using the module directive available in Blacklight's command line tool.

Each Blacklight site module has a unique ID consisting of two parts: a site id and a module id; for example my-site.my-area. As a developer, you create and extend your site by creating and editing Blacklight site modules. But Blacklight itself also uses site modules to implement key aspects of its own functionality. For example, the blacklight.edit module gives Blacklight the ability to work with edit dialogs, and the blacklight.docs module holds the documentation you are reading now. Code for any of your own site modules can be opened and edited in isolation from the rest, which helps prevent confusion with -- and side effects against -- code from the other modules. See Blacklight code organization for details.

Components

The concept of components is widely used in web development, and it might mean different things to different people. But in simple terms, a Blacklight component is generally a rectangle on screen that contains a logically distinct portion of page content and/or functionality. Usually any given instance of a component can be customized by feeding it data from the CMS and/or by its being sensitive to surrounding content. Components also can contain other components. In fact, in Blacklight the page itself is just another component, and it usually contains various types of child components.

Component File Paths and Resource Types

Each Blacklight component is stored in a single folder in your source tree. Most Sling content is tagged with a component type string -- called a resourceType in Sling parlance. Sling data is stored in a hierarchical NoSQL-like content repository. You might think of it as a very large XML document, in that there are parent and child nodes, any of which can be uniquely addressed with a URL-like path. In its basic form (without Blacklight), Sling will take browser requests on its own and will directly use the path in your browser's address bar as a key to determine which data node to retrieve.

Blacklight works very much the same way, passing address bar paths back to Sling to get an initial set of page data in JSON form. That page data will have a resourceType which itself maps to a path, but this time to a path to a component folder on disk. Here's a sample:

In the sample above, the my-site/my-area/pages/landing component has two functional pieces: 1) landing.js to do any model processing of the raw JSON that may be required, and 2) landing.hbs a template to render the final JSON into HTML. In addition to model processing and rendering, any component can also implement an edit dialog to let content managers change component data in the CMS. The next sections expand on these three component functions.

Component Templates

Blacklight uses the Handlebars templating language to generate HTML from the JSON that Sling returns. Handlebars comes with a number of built-in "helper" functions that are available to use in your Blacklight templates. Blacklight adds in a few custom helpers as well, in particular the following:

• component helper -- Takes a data node as an argument, looks at the data's associated resourceType, loads the appropriate component template, and runs that template against the data.
• img-opt helper -- Takes a Sling JPG image path and
• translate helper -- Which blah blah blah
• ifop and ifnot helpers -- Which blah blah blah

Component Model Processors

Component Edit Dialogs

• JSON data source / Sling
• JSON Model Processor
• Handlebars templates
• Modularized

Network Diagrams

The following diagram depicts the typical Blacklight configuration, including a Sling content repository fronted by a Blacklight instance:

Raw content data is stored in the Sling repository, which serves the JSON content to Blacklight via the restful Sling protocol. Depending on the page being rendered, Blacklight may apply some business logic to the JSON returned from Sling, aggregating and transforming it. Finally, Blacklight will render the JSON into finished HTML for delivery to the browser, possibly via a cache layer such as Nginx and/or a CDN such as CloudFlare.

When developing Blacklight modules locally (e.g. on a laptop), the "Blacklight-first" configuration shown above is frequently used, often with the Sling content repository being a remote staging instance which is shared by multiple Blacklight installations.

By contrast, in a production environment you might be more likely to see that order reversed, with Sling on the front end of the request pipeline and Blacklight on the back end, as depicted in the following diagram:

In this "Sling-first" type of configuration, a piece of software called the Blacklight Proxy is installed into Sling, and performs the task of deciding whether to render raw content from the repository entirely in Sling, or to send the raw JSON over to Blacklight to be rendered. The "Sling-first" approach is usually used when a site has already been running on a Sling-based CMS, and there is a desire to incorporate some Blacklight pages into the environment. In such a case, the Blacklight Proxy is installed and then configured to send certain types of requests to Blacklight, and to leave the rest for Sling to sort out.

For most Blacklight development work, it is best both to configure and to envision the system as depicted in diagram 1, i.e. the "Blacklight-first" approach.

Blacklight request handling

Blacklight is a regular Node.js Express application, and behaves much like any other Express app does. In particular, it uses the Express router to determine which methods(s) to invoke to handle a given request. In Blacklight, all route handlers lead to your site's index.js file, which is the main point of entry for your application (to illustrate this fact, you can type > node index.js from a command line to start your Blacklight server). The index.js file is generated by the > bl init utility, but is otherwise a fairly typical Express app. If you examine your index.js file you'll find a set of route handlers, usually in the following sequence:

1. img-opt handler -- Blacklight image optimizer service. http://www.mysite.com/apps/img-opt

2. Module-level "rooted" routes -- Can attach to any path from the root of the URL namespace. Use sparingly -- only when your project requires you conform to a URL convention different than Blacklight's preferred module name spacing. http://www.mysite.com/some-special-api

3. Module-level "regular" routes -- Can attach to any path under its module's namespace. http://www.mysite.com/apps/my-site/my-module/some-other-api

4. Static files handler -- Static files are served via the publicMount folder (usually set to assets). http://www.mysite.com/assets/my-site/my-module/some-file.js

5. Blacklight page renderer -- Finally, any request not handled by the above will be sent to the Blacklight page renderer. http://www.mysite.com/content/my-site/some-page.html

Note: Blacklight's generated index.js file can be edited by hand, and for certain tasks this is the simplest way to make changes to your Blacklight instance. However, it is recommended you do not edit the index.js file to add custom routes. The standard Blacklight route configuration API (which is available to every Blacklight module, and briefly described in points 2 and 3 above) provides ample facility to accommodate just about any need your site may have in terms of route namespacing. By configuring your routes using Blacklight conventions, your work remains more manageable and trackable than when editing index.js by hand.

Blacklight Page Rendering Pipeline

As described in step 5 above, Blacklight invokes its "page rendering pipeline" handler when it receives a request for a URL that is not handled by any other route configured in the Express stack. The Blacklight rendering pipeline is broken up into four main stages:

• Preprocessing
• Model processing
• Template Rendering
• HTML post-processing

The following diagram illustrates and expands on the four stages of the render pipeline:

Preprocessing directives

Blacklight's pre-processor directives are currently stored in an array called requestPreprocessors, which in turn resides in the main index.js file -- but these will likely move to the Blacklight module level at a future date. Each requestPreprocessor entry consists of a path or type filter, and a directive (currently, redirect, proxy, or slingPreprocess are available). The filter determines which requests the given directive should apply to. Here is an example configuration:

requestPreprocessors = [
	// Redirect requests for 'some-area' to 'some-other-area'
	{path: /^\/content\/my-site\/some-area\/(.*)$/, redirect: "/content/my-site/some-other-area/$1" },

	// Inject analytics data stubs for all mysite.com page requests
	{path:"^/content/my-site/", slingPreprocess: myAnalyticsPreprocessor},

	// Create a page path that gets its raw JSON from another source rather than Sling
	{path:"^/static-content/", proxy: myJSONproxy}
];

///////////////////////////////////////////////////////////////////////////////////////////////
function myAnalyticsPreprocessor(model, $, cb){
	if(!$.page.trimmedPath.match(/my-excluded-pattern/))
		model.analytics={_sling_resourceType:"my-site/shared/content/analytics", pageInfo={}}
	cb(null);
}

///////////////////////////////////////////////////////////////////////////////////////////////
function myJSONproxy(urlPath, $, cb){
	var path = require("path").join("/var/static-data", urlPath + ".json");
	fs.readFile(path, (err, contents){
		if(err){console.error("ERROR: Problem reading static JSON file:", path);}
		cb(err, contents);
	});
}

Here is a description of the three types of requestPreprocessor directives that are available:

1. redirect is the first type of preprocessor Blacklight will try to apply to a given page request, when the URL matches the directive's path filter (the type filter is not available at this stage because no data has been retrieved, and so the page type is not known). A redirect function can rewrite the incoming request URL, before Blacklight passes that URL on for futher processing in the pipeline.

2. proxy is the second type of preprocessor Blacklight will try to apply, based again on a match with the given path filter (and again, the type filter is not available at this stage because no data has yet been retrieved). A proxy function is able to bypass the normal Sling data retrieval functionality, and replace it with a completely customized JSON retrieval process. For example, you may wish for certain requests to pull data from a different Sling source than what is configured by default; or you may wish certain requests to retrieve JSON from some other data source that has nothing to do with Sling.

3. slingPreprocess is the final type of preprocessor directive that is considered. After applying redirect and proxy directives, Blacklight will apply any slingPreprocess directives that match the path or type specified. This final directive allows you to provide a function that can alter any previously retrieved JSON, in any way required. While this optional step is very similar to the much more prominent model processor stage of the render pipeline, the slingPreprocess step has the ability to add and alter resourceTypes to the base JSON retrieved for any page, something which Blacklight's main model processor is currently unable to do. In future, Blacklight's internals may be tweaked to allow resourceType manipulation in the main model processor, but until then slingPreprocess is the best way to do so.

Blacklight Code Organization & IDE Usage

Anatomy of a Blacklight Module Folder

The Blacklight module loader looks for four key folders in each site module:

1. components -- Templates, model processors and dialog definitions for each component in this module.
2. public -- Static assets (JS, CSS, image files, etc) referenced in public pages rendered by this module.
3. apps -- Custom API routes, model processor helpers, command line tools and other supporting services that might be needed by the module.
4. docs -- Documentation files relevant to this module, in Markdown format.

Soft-linked Module Folders & IDE Usage

One quirk of Blacklight folder layout is that, upon installation, three of the four folder types listed above are soft-link "mounted" into a second location in the top-level folder hierarchy. The diagram at the right illustrates where these secondary soft links reside.

This soft-linking accelerates and simplifies file lookup when Blacklight is running. For example, all module public folders get mapped into a contiguous set of module-specific folders under a top-level public umbrella folder, with a resulting structure that translates directly to a logical URL namespace. components and apps also benefit from this remapping, for similar reasons.

Although this soft-linking makes things easier for the Blacklight server, for developers it can ambiguate how best to approach the files when doing development. When using a folder-oriented IDE such as Sublime Text or Atom, the question becomes which folder to open in the IDE window, and then which folders to hide. Two primary options are worth considering when working in a folder-based IDE:

1. All files and modules in a single window -- For this option, looking at the example on the right, you'd open the entire "my-site" folder in your IDE window, and then use the IDE's left-nav file tree to work your way down to the particular file you need. For example, to edit the main template for the landing page component in the "my-area" site module, you'd drill down to: /my-site/components/my-site/area-one/pages/landing/landing.hbs. When taking this view on the files in your IDE, you will probably want to hide the blacklight_modules folder, so as not to get duplicate results when searching in files, etc.

2. Each module in a different window -- Again, using the example in the right-hand figure, and to edit the same file as in option 1, you'd first open only the relevant module folder in your IDE window: /my-site/blacklight_modules/my-site/area-one. Then from there, navigate in the IDE's left-nav file tree to your file: components/pages/landing/landing.hbs.

Option 2 has a few notable advantages over option 1: Notice the path to the example file in the tree is much shorter in option 2 than in option 1, which means less "drilling down" to get where you need. Also with option 2, there is no need to hide any folders in your IDE to get optimal search results (though just like with option 1, you may still wish to hide node_modules and any public/**/vendor folders). Another advantage of option 2 is it gives you a concise view of all component, public, and apps files, with much less confusing distance between them, and no files from unrelated modules cluttering the view.

Option 1 allows you to see three key files that are not visible from option 2: the main index.js, config/default.json and config/local.json. Option 1 also provides an easy way to search through all modules for certain stings or references from a single point. So even if, like the creators of Blacklight, you prefer using option 2 for most module-specific development, it is definitely useful to keep option 1 handy for certain types of changes.

So depending on how you like to conceptualize things, option 2 is mostly upside as compared to option 1. One of the only potential downsides you might find for option 2 is the need to jump between IDE project windows when you wish to work on another module; though arguably, even that has some advantages in terms of mental compartmentalization of concerns. For Sublime Text, the GotoWindow package provides a very nice keyboard shortcut for quickly jumping between multiple project windows, using super + shift + o.

Sling Repository

All about Sling. Mostly pointers to Apache Sling project.

Sling-to-Blacklight Proxy

Publish vs. Author repositories

Rendering Pages & Components

• Pages, Nodes, and Properties
• Components & Resource Types
• Components Folder
• Handlebars & Custom Helpers
• Model Processors

Dialogs, Containers & Widgets

• How to augment component folders with dialog.js and container.js
• Approving Content
• Modules
• Multilingual