Node Tuts

Detach script window ⇗

You can navigate the video and the script by using the ← and → cursor keys.

Streams

Node comes with this wonderful abstraction called Streams. There are two basic types of streams: Read Streams and Write Streams.

Read Stream

A read stream is a source of data: it behaves like a data faucet, emitting data events. An example of a read stream is a file read stream:

var fs = require('fs');

var readStream = fs.createReadStream('/var/log/system.log');

readStream.on('data', function(data) {
  console.log('got some data:', data);
});

readStream.on('end', function() {
  console.log('ended');
});

As you can see, a read stream (and a write stream) is a special case of an event emitter that emit the data and end event types.

Let's run this script.

As you can see, there are several data events being triggered, each with a chunk of data of the file that we're reading.

Setting The Encoding

Each data event gets a buffer as the data payload. If you specify an encoding, the buffers will be encoded into a string:

readStream.setEncoding('utf-8');

If you run this again, you will now get UTF-8-encoded strings.

There are other supported encodings like base64:

readStream.setEncoding('base64');

Flow Control

Just like a faucet, you can pause and resume a read stream. Let's try to just emmit one chunk of data each second:

var fs = require('fs');

var readStream = fs.createReadStream('/var/log/system.log');

readStream.pause();

readStream.on('data', function(data) {
  console.log('got some data:', data);
  readStream.pause();
  setTimeout(function() {
    readStream.resume();
  }, 1000);
});

readStream.on('end', function() {
  console.log('ended');
});

setTimeout(function() {
  readStream.resume();
}, 1000);

Let's run this and watch the effect on the log output.

Write Streams

A write stream is a stream that you write data to. An example of a write stream is a file write stream:

var fs = require('fs');

var writeStream = fs.createWriteStream('/tmp/write.txt');

var interval = setInterval(function() {
  writeStream.write((new Date()).toString() + '\n');
}, 100);

setTimeout(function() {
  clearInterval(interval);
  writeStream.end();
}, 5000);

Here we're creating a file write stream and writing a string to it every 100 miliseconds. After 5 seconds we're ending the stream.

Let's test this code.

Flow Control

Every time you write to a write stream, the data gets buffered until it is flushed to the kernel. Every time the Node buffers are depleted, you get a drain event. Let's see that in action by adding this line to the end of our example:

writeStream.on('drain', function() {
  console.log('drained.');
});

writeStream.write() returns a boolean that is true if the data was immediately flushed, and false if the data got buffered. Let's also observe this return value:

var interval = setInterval(function() {
  var flushed = writeStream.write((new Date()).toString() + '\n');
  console.log('flushed:', flushed);
}, 100);

Let's see this in action.

Pipe

A read stream can be piped into a write stream by using stream.pipe() like this:

readStream.pipe(writeStream);

Pipe also deals with back-pressure: when the write is not flushed, the read stream is paused until the write stream emits the drain event.

Let's now see some uses of pipe. For that let's create an HTTP Server:

var http = require('http');
var fs = require('fs');

function randomFileName() {
  return '/tmp/' + Date.now() + '_' + Math.random() * 1000000 + '.txt';
}

http.createServer(function(req, res) {
  var fileName = randomFileName();
  console.log('writing request to', fileName);

  req.pipe(fs.createWriteStream(fileName));

  res.writeHead(200, {'content-type': 'text/plain'});
  fs.createReadStream(__filename).pipe(res);
}).listen(8080);

Here we're creating an HTTP server listening on port 8080. On each request we get the request and response objects.

The request object is a read stream which we pipe to a file write stream.

We also pipe the current source file into the response — which is a write stream. we're piping the request body.