The art of gulp - sameeri/Code-gulp GitHub Wiki

gulp is a build tool. Just like rake is for the ruby community, MSBuild is for .NET world. If you have heard of make, ant, then you can imagine gulp in the same line of build automation tools.

If you have ever worked with statically compiled languages, you might have encountered a step after writing your code. That of compilation. Code when it is compiled gets to a binary code format, whether it's directly machine code or some intermediate code. And then your program would be available for use.

This was originally what someone would mean to say : to build - compile the source code of the project and get it to make ready for use! Building was then synonymous to 'is it compiling?'. Programs in the olden days, would just run on the same machine, either as Console or Graphical interfaces. But then, additional steps such as packaging the application exist.

Over the years different kinds of apps, some that run inside a browser(web apps.), others that run on mobile devices etc, programs that run inside appliances, have surfaced. Also, with different kinds of process and practices, (such as test driven development, feature driven development, automated user tests..), technology improvements(cloud services), the notion of a build process, has become very important in an application's development process and to organizations in general. This includes open source projects. Open source projects have the ability to let the community contribute to their project. A well defined build process helps the contributor. The state of the project's build is a crucial aspect. Many organizations expose to the world, their projects build status.

Loosely defined, a build process is a series of tasks that run to do different things like compilation, running tests, deployment etc. This process varies for each project and is defined by the individual/team.

The process can usually be automated. Build tools let you define these steps of a build process, and help you execute them. How you define a step, is an implementation detail of the build tool.

Two software principles come into play when you talk about build processes and build automation.

A project build should require one step.

Fail quickly

These are best practices that a software team would follow.

• A build process would be defined and setup using a build tool. This depends on the application in development. For example, for a web app, one might want to do a JSHint, minify JS, optimize static resources such as images etc. The practice of optimizing various resources, minimizing the number of http calls is a relatively new thing to the web development community and has been prescribed by Steve Souders & his performance team at Yahoo. Steve now works for Google and they discuss web performance exclusively here.
• Build types such as development, staging, production might be setup differently and maybe different for the same kind of step. For instance, the deployment step in a build process would probably deploy to different servers.
• A Continuous Integration environment is setup so that anyone in the team or organization knows the status of a build. The CI server generates a report of the builds.

Since we are in the context of gulp, it would be nice to have an idea of what things tend to get automated in the JS world:

• JS hint
• Unit tests
• Integration tests
• Behavior tests
• Performance tests
• Coverage analysis
• Code complexity
• Deployment

These are routine and are applicable to all kinds of projects.

In the case of web apps,

• Code minification
• Image optimizations
• Running your styles through a preprocessor etc..

gulp is from the JavaScript world. The JavaScript community had already seen grunt as a build tool. And it is used by many. However, even with grunt around, gulp was created. This stems from a philosophical difference of opinions.

grunt like many other build tools uses configuration as a means of defining a series of steps. Build tools like ant, MSBuild use XML configuration to define a build process. grunt uses json.

gulp on the other hand uses code to define it's build process. You write code telling what gulp should do.

Some people prefer code. Some people prefer configuration.

At the end of the day, it's important to have a build process, have it automated using a tool of the team's choice.

If you spend some time around the gulp project, you might run into this interesting issue.

• https://github.com/gulpjs/gulp/issues/540

Tasks are a common find in the vocabulary of build tools. A task specifies a step or a series of steps. Tasks are defined and a build tool is coded/configured to run tasks in a certain way!

gulp, rake, grunt.. are essentially task runners. They let you define tasks. And when you invoke build tool specifying what task to run, it runs the task for you. Also, it's not essential that you have a complete build process. It can be simple tasks that would help you in your everyday work to do things more smoothly, to set up a workflow. The essence of task runners is to help you automate.

For example, a scaffolding utility slush has been written that uses the power of gulp. [Some people opine that scaffolding is a part of the build process, some consider it not a part. It all depends on how you look at it. :)]

A simple task could be to run your unit tests. You would go and define such a task. You would say in the task, go find my tests here, and run them using my favorite test runner/framework. This task may be then invoked by itself, or may find itself being a step in a complete build process.

To get started let's first read docs from gulp.

We shall also read some [gulp code] as we try to understand gulp.

gulp like many other build tools is provided as a CLI.

To run gulp, first we need to install gulp. It's available as a node module, and so we can use npm.

$ npm install -g gulp
C:\Users\sameeri\AppData\Roaming\npm\gulp -> C:\Users\sameeri\AppData\Roaming\npm\node_modules\gulp\bin\gulp.js
[email protected] C:\Users\sameeri\AppData\Roaming\npm\node_modules\gulp
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected]
├── [email protected] ([email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
└── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected])

Awesome! Now, try to run gulp. It's as easy as saying gulp.

$ gulp
[18:32:05] No gulpfile found

gulp spits an error. It tries to find a special file called gulpfile in your working directory. If you have ever worked with rake then rake expects to find a rakefile.rb. Same with gulp. Sadly, i do not have such a file in my current directory. Let's go ahead and create a gulpfile and run gulp again.

$ gulp
[18:38:01] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js
[18:38:01] Task 'default' is not in your gulpfile
[18:38:01] Please check the documentation for proper gulpfile formatting

This time we have another issue. gulp could find gulpfile but couldnot find a task called default in the gulpfile. We will fix this in just a second.

To understand how to create a task, we will first try to understand gulp's API.

gulp has two parts. A module that you can require in your code and work with gulp tasks, and a CLI that knows how to execute your gulp based tasks.

• https://github.com/gulpjs/gulp/blob/master/index.js - Node module that you can require and work off. gulp API.
• https://github.com/gulpjs/gulp/blob/master/bin/gulp.js - CLI that helps run gulp tasks.

So, we need to require gulp in our gulpfile.

var gulp = require('gulp');
console.log('gulp:', gulp);

If we log the gulp we find it to be a simple object and there are no functions that we can see.

$ gulp
gulp: { domain: null,
  _events: {},
  _maxListeners: undefined,
  doneCallback: undefined,
  seq: [],
  tasks: {},
  isRunning: false }

Looking at gulp's code, we find that gulp is an instance object returned from the Constructor function Gulp.

function Gulp() {
  Orchestrator.call(this);
}
util.inherits(Gulp, Orchestrator);

//somewhere at the bottom of the file
var inst = new Gulp();
module.exports = inst;

Okay! Let's try to see the prototype of this constructor function and what methods it has to offer.

console.log('gulp - constructor', gulp.constructor);
console.log('gulp - constructor prototype', gulp.constructor.prototype);

gulp - constructor function Gulp() {
  Orchestrator.call(this);
}
gulp - constructor prototype { task: [Function],
  run: [Function],
  src: [Function: src],
  dest: [Function: dest],
  watch: [Function],
  Gulp: { [Function: Gulp] super_: { [Function] super_: [Object] } },
  env: [Getter/Setter] }

As we have seen, gulp has a pretty simple API.

That is all.

From the constructor function, we have observed that gulp is inheriting from Orchestrator. Infact, gulp is a wrapper over Orchestrator. It exposes Orchestrator's methods add and start. add helps in task creation, and start runs the tasks. We can create a map of Orchestrator & gulp.

Awesome! Let's go write our first task.

To define/create a task in gulp is very easy. Use the task api.

gulp.task('default', function doSomething(){
  console.log("This is default. I am important for gulp tried to find me if you don't specify a task!");
});

Task api let's you define a task by giving it a name and specifying a callback function. The function tells gulp what to do when it executes that task.

Now when we run gulp it finds a "default task" and it executes it.

$ gulp
[19:13:05] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js
[19:13:05] Starting 'default'...
This is default. I am important for gulp tried to find me if you don't specify a task!
[19:13:05] Finished 'default' after 185 μs

Add another task.

gulp.task('Awesome', function (){
  console.log('Taking gulp awesomeness in steps!');
});

How can we run this?

It's very easy. We can specify one or more tasks as arguments for gulp.

gulp <task1> [...].

$ gulp Awesome
[19:14:23] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js
[19:14:23] Starting 'Awesome'...
Taking gulp awesomeness in steps!
[19:14:23] Finished 'Awesome' after 195 μs

sameeri@SMARRYBOYINA /c/sameeri/Code-gulp (start)
$ gulp Awesome default
[19:14:35] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js
[19:14:35] Starting 'Awesome'...
Taking gulp awesomeness in steps!
[19:14:35] Finished 'Awesome' after 177 μs
[19:14:35] Starting 'default'...
This is default. I am important for gulp tried to find me if you don't specify a task!
[19:14:35] Finished 'default' after 102 μs

These are the most useful ones. There are others

$ gulp --version
[19:16:32] CLI version 3.9.0
[19:16:32] Local version 3.9.0

sameeri@SMARRYBOYINA /c/sameeri/Code-gulp (start)
$ gulp -T
[19:16:38] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js
[19:16:38] Tasks for c:\sameeri\Code-gulp\gulpfile.js
[19:16:38] ├── default
[19:16:38] └── Awesome

In gulp a task can be composed of some other task(s).

gulp.task('A', ['B']);

When we run task A, we run task B. A serves as an alias in this case, just another definition for task B. But A can be a task that is a composition of multiple tasks.

gulp.task('A', ['B', 'C', 'D']);

When we run A, the tasks B, C, D execute asynchronously.

What if we want some tasks to be execute synchronously? We could use the popular helper module, run-sequence

gulp.task('A', function(callback) {
  runSequence('B',
              ['C', 'D'],
              'E',
              callback);
});

Running task A runs B, C, D, and E. It's gauranteed that B runs first. After it finishes, C, D run parallely(asynchronously) and when both of them finish it runs E.

Also, we can pass in a callback to runSequence, to do something after everything finishes. Quite handy!

What if we want to depend on some task(s), and then do something?

It's simple, we can use the task API to pass in a callback after we mention the dependent tasks.

gulp.task('A', 'B', function(callback) {
  runSequence('B',
              ['C', 'D'],
              'E',
              callback);
});

• https://github.com/gulpjs/gulp/issues/96

Let's take a look at a couple of examples.

Quite recently i learnt how to make gaucamole. And i can't believe that it's so simple to make some nice tasty gauc.

The recipe i follow is:

Chop onions, tomatoes, chillies(serrano,peppers), scoop out the inside of avacado. Put it all in a bowl and mash it up together. Add some gaucomole mix that you can get at the local store, cilantro, and lime, salt for flavor.

We would like to try mimic this procedure using gulp.

We can divide our gaucamole recipe into 3 parts. Ingredient preparation, mixing it all up and then adding final flavor based ingredients.

var gulp = require('gulp');
var runSequence = require('run-sequence');

gulp.task('Cut Onions', function(){
    console.log('Cutting oninons!');
});


gulp.task('Cut Tomatoes', function(){
    console.log('Cutting tomatoes!');
});

gulp.task('Cut Peppers', function(){
    console.log('Cutting peppers!');
});

gulp.task('Scoop Avacado', function(){
    console.log('Avacadooo!');
});

gulp.task('Mash n Mix', function(){
    console.log('Mix everything!');
});


gulp.task('Add Cilantro', function(){
    console.log('Add some flavor!');
});


gulp.task('Add gauc mix', function(){
      console.log('Some spicy gaucamole mix :)');        
});


gulp.task('Add lime', function(){
    console.log('Nice and sour!');
});

gulp.task('Prepare ingredients', ['Cut Onions', 'Cut Tomatoes', 'Cut Peppers', 'Scoop Avacado']);

gulp.task('Add extras', ['Add Cilantro', 'Add gauc mix', 'Add lime']);

gulp.task('Recipe', function (){
    runSequence('Prepare ingredients','Mash n Mix', 'Add extras'); 
    console.log('We have amazing gauc ready for you!');
});

Let's first list out all the tasks.

$ gulp -T
[21:13:58] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js
[21:13:58] Tasks for c:\sameeri\Code-gulp\gulpfile.js
[21:13:58] ├── Cut Onions
[21:13:58] ├── Cut Tomatoes
[21:13:58] ├── Cut Peppers
[21:13:58] ├── Scoop Avacado
[21:13:58] ├── Mash n Mix
[21:13:58] ├── Add Cilantro
[21:13:58] ├── Add gauc mix
[21:13:58] ├── Add lime
[21:13:58] ├─┬ Prepare ingredients
[21:13:58] │ ├── Cut Onions
[21:13:58] │ ├── Cut Tomatoes
[21:13:58] │ ├── Cut Peppers
[21:13:58] │ └── Scoop Avacado
[21:13:58] ├─┬ Add extras
[21:13:58] │ ├── Add Cilantro
[21:13:58] │ ├── Add gauc mix
[21:13:58] │ └── Add lime
[21:13:58] └── Recipe

Let's run our code.

$ gulp Recipe                                                      
[21:14:40] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js         
[21:14:40] Starting 'Recipe'...                                    
[21:14:40] Starting 'Cut Onions'...                                
Cutting oninons!                                                   
[21:14:40] Finished 'Cut Onions' after 182 μs                      
[21:14:40] Starting 'Cut Tomatoes'...                              
Cutting tomatoes!                                                  
[21:14:40] Finished 'Cut Tomatoes' after 109 μs                    
[21:14:40] Starting 'Cut Peppers'...                               
Cutting peppers!                                                   
[21:14:40] Finished 'Cut Peppers' after 86 μs                      
[21:14:40] Starting 'Scoop Avacado'...                             
Avacadooo!                                                         
[21:14:40] Finished 'Scoop Avacado' after 93 μs                    
[21:14:40] Starting 'Prepare ingredients'...                       
[21:14:40] Finished 'Prepare ingredients' after 3.69 μs            
[21:14:40] Starting 'Mash n Mix'...                                
Mix everything!                                                    
[21:14:40] Finished 'Mash n Mix' after 85 μs                       
[21:14:40] Starting 'Add Cilantro'...                              
Add some flavor!                                                   
[21:14:40] Finished 'Add Cilantro' after 87 μs                     
[21:14:40] Starting 'Add gauc mix'...                              
Some spicy gaucamole mix :)                                        
[21:14:40] Finished 'Add gauc mix' after 119 μs                    
[21:14:40] Starting 'Add lime'...                                  
Nice and sour!                                                     
[21:14:40] Finished 'Add lime' after 135 μs                        
[21:14:40] Starting 'Add extras'...                                
[21:14:40] Finished 'Add extras' after 3.69 μs   
We have amazing gauc ready for you!                  
[21:14:40] Finished 'Recipe' after 16 ms                           
                                                                   

Nice!

We don't care if the ingredients are prepared in a specific order. They can happen in parallel, or however it is convenient. The same with extras. The only thing that we know is only after getting everything ready, we can mash them up. So there is a requirement on task execution order here.

gulp is a build tool that uses code to specify the tasks of a build process. The creators of the tool, We are fractal found it very difficult to work with grunt's configuration, so they created gulp.

Since gulpfile contains code, it should be treated with the same respect as your production code. Maybe, even more so, because it defines your process and the way you do things.

Having clean code is necessary as code is for people.

Programs are meant to be read by humans and only incidentally for computers to execute. — Donald Knuth

Going forward with our code, we shall try to avoid anonymous functions, have a pattern in place and things like that!

Let us continue in our pursuit of gulp glory and come with a sample build process.

Let's say we are working with JS, and our project is an API, using some popular framework such as hapi or express.

We would probably like to do the following:

• Run JSHint on source and test files jshint
• Run unit tests mocha
• Run integration tests 'cucumber-js`
• Run code coverage analysis istanbul
• Run code complexity analysis using something like plato
• Generate API Docs
• Deploy to a server.

Also, let's say we have different environments, such as develop, staging and production.

Although an API runs on a server and we don't need to minify JS code, in a web app, for instance we would like to minify JS code that gets sent to the browser. The point to be taken home is for different environments, we might want to run different set of tasks!

var gulp = require('gulp');
var gutil = require('gulp-util');
var runSequence = require('run-sequence');


function runStaticCodeAnalysis(){
    console.log('Run JShint on all the JS files, code & tests');
    console.log('All our code will look consistent, hopefully!');
}

function runUnitTests(){
    console.log('Mocha/Jasmine');
    console.log('Code should have unit tests! Make sure that all tests pass');
}

function runIntegrationTests(){
    console.log('Hopefully Cucumber.js');
    console.log('Since we develop features, our Specs should have tests and they should pass');
}

function coverageMetrics(){
    console.log('Istanbul');
    console.log('With code coverage, what code is covered with our unit tests and what is not.');
}

function runComplexityAnalysis(){
    console.log('plato');
    console.log('With code complexity, we shall see how well we"ve written code');
}

function minify(){
    console.log('jsminify');
    console.log('Let"s say this is a build file for a library! We would need two versions. Development. Production. Production should have minified JS');
}

function deploy(){
     console.log('Deployment mechanisms');
     console.log('Deploy to whereever! CDNs maybe?');
}

function generateApiDocumentation(){
    console.log('Some doc generator');
    console.log('Everyone loves our library. Give them good docs for API Usage!');
}

function runDevBuild(){
    gutil.log('Running DEV build');
    runSequence('Static Code Analysis', 'Unit Tests', 'Integration Tests', 'Coverage', 'Complexity Ananlysis', 'Generate Documentation', 'Deploy');
}


function runStagingBuild(){
    gutil.log('Running STAGING build');
    runSequence('Static Code Analysis', 'Unit Tests', 'Integration Tests', 'Coverage', 'Complexity Ananlysis', 'Generate Documentation', 'Minification', 'Deploy');
}


function runProductionBuild(){
    gutil.log('Running PRODUCTION build');
    runSequence('Static Code Analysis', 'Unit Tests', 'Integration Tests', 'Coverage', 'Complexity Ananlysis', 'Generate Documentation', 'Minification', 'Deploy');
}

gulp.task('Static Code Analysis', runStaticCodeAnalysis);

gulp.task('Unit Tests', runUnitTests);

gulp.task('Integration Tests', runIntegrationTests);

gulp.task('Coverage', coverageMetrics);

gulp.task('Complexity Ananlysis', runComplexityAnalysis);

gulp.task('Minification', minify);

gulp.task('Generate Documentation', generateApiDocumentation);

gulp.task('Deploy', deploy);

gulp.task('build:develop', runDevBuild);

gulp.task('build:staging', runStagingBuild);

gulp.task('build:production', runProductionBuild);

gulp.task('default', ['build:develop']);

• We have rearranged our code unlike the gaucamole code.
• All our functions are pulled out to the top, and all our tasks are single line definitions now.
• We have also sort of namespaced our builds as build:dev and so on.
• We have also set up the dev run as our default task so when someone says gulp, it runs develop build.
• Each task could be also further pulled into it's own file thus creating a library of tasks that are reusable across multiple projects.

Now, we can run the staging build, for instance.

$ gulp build:staging
[10:34:54] Using gulpfile c:\sameeri\Code-gulp\gulpfile.js
[10:34:54] Starting 'build:staging'...
[10:34:54] Running STAGING build
[10:34:54] Starting 'Static Code Analysis'...
Run JShint on all the JS files, code & tests
All our code will look consistent, hopefully!
[10:34:54] Finished 'Static Code Analysis' after 261 μs
[10:34:54] Starting 'Unit Tests'...
Mocha/Jasmine
Code should have unit tests! Make sure that all tests pass
[10:34:54] Finished 'Unit Tests' after 198 μs
[10:34:54] Starting 'Integration Tests'...
Hopefully Cucumber.js
Since we develop features, our Specs should have tests and they should pass
[10:34:54] Finished 'Integration Tests' after 183 μs
[10:34:54] Starting 'Coverage'...
Istanbul
With code coverage, what code is covered with our unit tests and what is not.
[10:34:54] Finished 'Coverage' after 192 μs
[10:34:54] Starting 'Complexity Ananlysis'...
plato
With code complexity, we shall see how well we"ve written code
[10:34:54] Finished 'Complexity Ananlysis' after 174 μs
[10:34:54] Starting 'Generate Documentation'...
Some doc generator
Everyone loves our library. Give them good docs for API Usage!
[10:34:54] Finished 'Generate Documentation' after 175 μs
[10:34:54] Starting 'Minification'...
jsminify
Let"s say this is a build file for a library! We would need two versions. Development. Production. Production should have minified JS
[10:34:54] Finished 'Minification' after 244 μs
[10:34:54] Starting 'Deploy'...
Deployment mechanisms
Deploy to whereever! CDNs maybe?
[10:34:54] Finished 'Deploy' after 181 μs
[10:34:54] Finished 'build:staging' after 14 ms

NOTE: We could have more tasks in our process, like:

• Cleaning up environment (remove node modules, cache clean)
• Setting up the environment

defined, which are common and valuable.

So far we have created tasks, and have looked at task dependencies, execution order. In all our tasks, we have only used console.log to observe task execution. We have not done anything useful, or productive.

We should move on to the next two api methods on gulp. src and dest. These are very powerful, the meat of gulp, and bring to us many concepts.

Looking at gulp's source again,

var vfs = require('vinyl-fs');

Gulp.prototype.src = vfs.src;
Gulp.prototype.dest = vfs.dest;

the src and dest methods are just referencing the vinyl-fs module's src and dest methods. We will get back to vinyl-fs in a bit here, but first, let's use the methods exported from it.

The glob concept comes from the Unix world. The initial versions of Unix made available a command called glob, and also a function glob() in the shell programming domain. This was a way of expressing patterns. This is a very powerful technique.

Environments like shells, and language libraries support globs and pattern matching.

In the JS & the node world, there exists such a library for globbing support. node-glob.

As the name suggests, the src method takes in globs and gets the files, and the dest method writes to the path specified. If the folder doesn't exist it tries to create the folder.

We shall try to take the package.json and write it to a build folder.

var gulp = require('gulp');

gulp.task('default', function (){
      gulp.src('package.json')
          .pipe(gulp.dest('build');
});

When we run gulp, it runs our default task and writes the package.json file to the build folder.

There's a new method being used here, pipe. This brings us to two important concepts. Streams and Pipes.

gulp is the streaming build system y'all.

Streams, Stream libraries and APIs in languages, and Piping are not new concepts at all. They stem from the Unix philosophy and are seen in action in filter programs @ the termial.

Streams deal with data. They make data available in chunks when available instead of waiting for the complete set. They offer an abstraction over different data sources such as Files, I/O, Network etc. Programs deal with reading from and writing to streams. When you send data through such a set of programs, you are piping. You can form a pipeline by sending streaming data through such a set of programs by chaining them together.

Every programming language that supports streams might have the notion/api of a pipe(). This method is usually available to make data available to the next module that deals with stream data.

File Streams, I/O streams and Network streams are usually supported in many programming languages.

Node also has the support of Streams. A stream is an abstract interface implemented by various objects in Node. You can require the stream module when creating your own streams and based on the stream type you want to create, implement the methods and emit the right events. All streams are instances of the EventEmitter class in Node.

There are 4 types of streams in Node.

• Readable
• Writable
• Duplex (Readable, Writable)
• Transform (Duplex streams that manipulate data before sending it out)

It's important to realize that when you work with an existing streams such as the request, response objects of the http server, you are a consumer of the streams.

If you are creating something on your own and are trying to make a stream available to users, then your are a stream implementor.

When working with streams, it's important to understand the events that get generated. For example, a readable stream emits data and end events indicating that data is available and all data has been fetched.

Streams operate on Buffers, Strings but can also operate on JavaScript Objects in Object mode

A great resource for working node streams is the Stream Handbook

through is a wrapper module over the node's stream module.

through2 is inspired by through and is made available for streams 2

These modules make it easier to create streams.

The fractal guys have created a couple of concepts. Vinyl objects that describe files. And an adapter over the file system called [vinyl-fs][https://github.com/wearefractal/vinyl-fs].

There are 3 methods exposed via vinyl-fs. These are src, dest and watch.

The src method takes in glob(s) and uses through2, glob-stream to create streams the operate in object mode and return them. They return the object stream.

The dest method takes in a path. Writes actual files to the path, and also returns the stream.

gulp exposes the vinyl-fs src, dest and watch methods with the same names.

Coming back to our code example, using the src, dest methods, we can deal with actual files and use them in a pipe line.

gulp.src(globs)
    .pipe(gulp.dest(path));

The beauty of gulp lies in it's ability to work with files as streams. Since we are always dealing with streams, we can write small programs that operate on these streams and take some action and pass it along to another program.

These programs can be anything. It's up to our imagination. The only constraint is they they operate on vinyl file object streams. They should take in and send out these objects, that's all.

In the gulp ecosystem, these programs are called gulp-plugins. A catalog of plugins based on necessity can be found in the gulp-plugin registry

All these plugins are programs. They deal with streams. They deal with streams in object mode. That of vinyl file objects. As simple as that. They are transform streams. That is they are both readable & writable.

We can get a lot of insight from these two amazing resources on how to write gulp plugins.

• sindresorhus' gulp plugins
• Fractal's gulp plugins

Given a set of files, we do something with them and then pass them along. - gulp plugins

Using gulp plugins is a simple affair. They fit as building blocks of a gulp pipeline.

gulp.task('blah', function (){
var globs = '*.js'; //some glob pattern
var temp = 'temp'; //some intermediary path
var path = 'build'; //some path to write to
gulp.src(globs)
    .pipe(gulp-plugin1())
    .pipe(gulp-plugin2())
    .pipe(gulp-plugin3())
    .dest(temp)
    .pipe(gulp-plugin4())
    .dest(path);
});

Since every gulp plugin works with vinyl file object streams, we can pipe these together, to do something useful. This is a powerful technique. Imagine a conveyor belt. You pass the same thing a little modified in each stage. Each gulp plugin is designed to do that. Be Single responsible They can fit together and files flow through them.

Let us see a couple of examples from the sample gulpfile in gulp docs.

var imagemin = require('gulp-imagemin');
var paths = {
  scripts: ['client/js/**/*.coffee', '!client/external/**/*.coffee'],
  images: 'client/img/**/*'
};

// Copy all static images
gulp.task('images', ['clean'], function() {
  return gulp.src(paths.images)
    // Pass in options to the task
    .pipe(imagemin({optimizationLevel: 5}))
    .pipe(gulp.dest('build/img'));
});

The above example takes in some image files and runs it through an image minification plugin, gulp-imagemin.

var coffee = require('gulp-coffee');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var sourcemaps = require('gulp-sourcemaps');

gulp.task('scripts', ['clean'], function() {
  // Minify and copy all JavaScript (except vendor scripts)
  // with sourcemaps all the way down
  return gulp.src(paths.scripts)
    .pipe(sourcemaps.init())
      .pipe(coffee())
      .pipe(uglify())
      .pipe(concat('all.min.js'))
    .pipe(sourcemaps.write())
    .pipe(gulp.dest('build/js'));
});

This example uses a set of plugins and forms a pipeline. All the JavaScript files are first initialized with sourcemaps, and then sent through the gulp-coffee plugin, then minified using gulp-uglify and written to one single file. This file is written to the build/js folder using the dest api.

We should keep build tasks simple and make them do one thing. So it's good if we depend on one plugin to do this. For example, we should have a task for running jshint on all the source, test files. We should have one task for running tests. Keeping tasks single responsible makes it easy to work with them and also set up a well defined build process. We could collect a set of such tasks and keep them in a library and reuse them as needed. :)

Saying all that, it all depends on the task at hand. The task might need to depend on multiple plugins and make use of the powerful pipeline technique.

Sometimes, our task at hand might be so simple, that we do not even need to depend on gulp plugins. We could just node modules to In fact, a gulp task needs to exist if it does file transforms. Deleting a build folder does not need a gulp-plugin. We can depend on a node module to do that.

We can create a library of reusable tasks to be used whenever we need them. A lot of such libraries can be found on npm. We could form a collection of common tasks.

A most common task in JavaScript applications is to run them through a linter. Let's go ahead and create such a task. We can depend on the gulp-jshint plugin to do this.

var gulp = require('gulp');
var jshint = require('gulp-jshint');

var globs = 'src/*/*js';

function jsHint(){
 return gulp.src(globs)
    .pipe(jshint()); 
 }
}
gulp.task('jshint', jsHint);

Mocha is a great unit testing framework that runs on Node and in the browser. Given your practices, unit tests and automating their runs might be essential.

gulp-mocha is the gulp plugin we would like to use.

var gulp = require('gulp');
var mocha = require('gulp-mocha');

gulp.task('tests', function () {
    return gulp.src('tests/**/*.js', {read: false})
        .pipe(mocha({reporter: 'nyan'}));
});

Pretty simple. Also, the gulp-mocha plugin has ways to pass in options.