Third-party Modules

In the last episode you learned how you can use a Node Core module and also how you can create and use your own local modules.

But there is a whole world out there of open-source modules that other people have created waiting for you to use and abuse.

How Node Uses node_modules

When you require a module and you don't use a relative or absolute path, Node first searches for that string in the list of core modules like in this example:

var http = require('http');

But Node also looks inside a special local directory named node_modules — if it exists. Let's make an experiment:

$ mkdir -p node_modules/my_module

Create this file inside node_modules/my_module/index.js:

module.exports = 'MY_MODULE';

Then, at the root folder, create a file named test.js:

var my_module = require('my_module');
console.log(my_module);

And execute that file:

$ node test.js
MY_MODULE

What happened here? Looks like Node picked up our module from the node_modules folder. To prove that let's rename that folder:

$ mv node_modules zope_modules
$ node test.js
module.js:340

    throw err;
          ^
Error: Cannot find module 'my_module'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:362:17)
    at require (module.js:378:17)
    at Object.<anonymous> (/Users/pedroteixeira/projects/publications/nodetutsv2/code/06-third-party-modules/test.js:1:79)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.runMain (module.js:492:10)

Let's rename it back:

$ mv zope_modules node_modules

Managing sub-dependencies

This is fine and dandy, but what if my_module depended on another module? Say that my_module depends on a module named my_other_module. Now what?

Changing node_modules/my_module/index.js to introduce this dependency:

var myOtherModule = require('my_other_module');
console.log(myOtherModule);

module.exports = 'MY_MODULE';

Let's try and execute our test script again:

$ node test.js

module.js:340
    throw err;
          ^
Error: Cannot find module 'my_other_module'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:362:17)
    at require (module.js:378:17)
    at Object.<anonymous> (/Users/pedroteixeira/projects/publications/nodetutsv2/code/06-third-party-modules/node_modules/my_module/index.js:1:83)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:362:17)

That was expected. Let's try and meet that dependency by creating the file node_modules/my_other_module/index.js:

module.exports = 'MY_OTHER_MODULE';

Let's now try running our test script again:

MY_OTHER_MODULE
MY_MODULE

It worked, nice!

Except that my_other_module is not a direct dependency of ou test, it's a dependency of my_module. Let's do this right then:

$ mkdir -p node_modules/my_module/node_modules
$ mv node_modules/my_other_module node_modules/my_module/node_modules
$ tree
.
├── node_modules
│   └── my_module
│       ├── index.js
│       └── node_modules
│           └── my_other_module
│               └── index.js
└── test.js

$ node test.js
MY_OTHER_MODULE
MY_MODULE

There you go!

Introducing NPM

NPM — which stands as Node Package Manager — is the way to search, download, install and manage third-party modules.

Using NPM To Search For a Module

NPM is a searchable repository that you can get access to using the web interface from http://npmjs.org

You can also use the command-line npm to search, for instance, for CouchDB-related modyles

$ npm search couchdb
...

Installing a Module Using NPM

Let's try installing a module from NPM:

$ npm install request
npm http GET https://registry.npmjs.org/request
npm http 200 https://registry.npmjs.org/request
npm http GET https://registry.npmjs.org/request/-/request-2.12.0.tgz
npm http 200 https://registry.npmjs.org/request/-/request-2.12.0.tgz
request@2.12.0 node_modules/request

We can see that NPM has installed the request module inside the node_modules folder:

$ ls -la node_modules
total 0
drwxr-xr-x   4 pedroteixeira  staff  136 Nov 21 10:37 .
drwxr-xr-x   4 pedroteixeira  staff  136 Nov 21 10:08 ..
drwxr-xr-x   4 pedroteixeira  staff  136 Nov 21 10:25 my_module
drwxr-xr-x  14 pedroteixeira  staff  476 Nov 21 10:37 request

Here he is. Let's try and require him from our test script:

var request = require('request');

$ node test.js
MY_OTHER_MODULE
MY_MODULE
{ [Function: request]
  initParams: [Function: initParams],
  defaults: [Function],
  forever: [Function],
  get: [Circular],
  post: [Function],
  put: [Function],
  head: [Function],
  del: [Function],
  jar: [Function],
  cookie: [Function] }

Managing Globally Installed Modules

The command npm install downloads and install modules inside the local node_modules folder, but there is a global mode in NPM that you can activate with the -g flag.

Some modules are meant to be installed globally, like the mocha module:

$ npm install -g mocha
...
/usr/local/bin/mocha -> /usr/local/lib/node_modules/mocha/bin/mocha

Node modules may have executables bundled with them, and mocha is one of them. Now you should have the mocha executable in your path:

$ mocha --help

Also, the modules that are globally installed are available when you do a require inside any module, but the local modules take precedence.

As a rule of thumb, avoid installing a module globally.

Using package.json To Manage Your Dependencies

When you create a module or an application you can specify and fix the dependencies by having them in a manifest file named package.json:

{
  "name": "test",
  "version": "0.1.0",
  "dependencies": {
    "request": "*",
    "async": "*"
  }
}

This is specifying that our module / application depends on 2 packages: "request" and "async", any version.

You can then use NPM to update the depdencies:

$ npm update
...

Let's now inspect the local depdendency tree:

$ npm ls
test@0.1.0 /Users/pedroteixeira/projects/publications/nodetutsv2/code/06-third-party-modules
├── async@0.1.22
└─┬ request@2.12.0
  ├─┬ form-data@0.0.4
  │ ├── async@0.1.22
  │ └─┬ combined-stream@0.0.3
  │   └── delayed-stream@0.0.5
  └── mime@1.2.7

You can also pin down the version by being more specific about it inside the package.json manifest file:

{
  "name": "test",
  "version": "0.1.0",
  "dependencies": {
    "request": "2.12.x",
    "async": "0.1.x"
  }
}

Here we're saying that we want any patch version of version 2.12 of request and any patch version of async version 0.1.

You can use the npm update to see if there is a latest version that matches that specification and update it accordingly by running npm update:

$ npm update

For instance, if you wanted to downgrade to version 2.11 of request you could change the dependency specification accordingly:

{
  "name": "test",
  "version": "0.1.0",
  "dependencies": {
    "request": "2.11.x",
    "async": "0.1.x"
  }
}

And then run npm update again:

$ npm update

$ npm ls
test@0.1.0 /Users/pedroteixeira/projects/publications/nodetutsv2/code/06-third-party-modules
├── async@0.1.22
└─┬ request@2.11.4
  ├─┬ form-data@0.0.3
  │ ├── async@0.1.9
  │ └─┬ combined-stream@0.0.3
  │   └── delayed-stream@0.0.5
  └── mime@1.2.7

Summary

In this episode we analyzed how Node uses the node_modules folder to search for packages.

We used NPM to manage local and global packages.

We learned how we can use the package.json manifest file to pin down our dependencies and have NPM resolve them for us.