Buffers in Node – Part 1

Pure javascript, while great with unicode-encoded strings, does not handle straight binary data very well. This is fine on the browser, where most data is in the form of strings. However, node.js servers have to also deal with TCP streams and reading and writing to the filesystem, both which make it necessary to deal with purely binary streams of data. Node has several strategies for manipulating, creating, and consuming streams.

Raw data is stored in instances of the Buffer class. A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. A Buffer cannot be resized.

The Buffer class is a global, making it very rare that one would need to ever require('buffer').

Converting between Buffers and JavaScript string objects requires an explicit encoding method. The encodings that are supported are 'ascii', 'utf8', 'usc2', 'base64', 'binary' and 'hex'.

Writing to buffers

new Buffer(size) methods allocates a new buffer of size octets. Once the buffer is created, the buffer.write method is used to write a string value to the buffer

var buffer = new Buffer(100);

var stringToWrite = 'Sample text to add to a buffer';

//Write data to a buffer

buffer.write(stringToWrite, 'utf-8');

//finding the length of buffer

var offset = Buffer.byteLength(stringToWrite);

console.log(offset);

//Appending data to the existing buffer

buffer.write('. This is an appended text', offset, 'utf-8');

console.log(buffer.toString('utf-8'));

Copy values to buffer

buffer.copy allows one to copy the contents of one buffer onto another. The first argument is the target buffer on which to copy the contents of buffer, and the rest of the arguments allow for copying only a subsection of the source buffer to somewhere in the middle of the target buffer. For example:

var helloBuffer = new Buffer(50);

helloBuffer.write('Hello ', 0, 'utf-8')

var welcomeBuffer = new Buffer('node.js.');

var helloStringLength = Buffer.byteLength('Hello ');

var welcomeStringLength = Buffer.byteLength('node.js.');

//Copies from welcomeBuffer to helloBuffer

welcomeBuffer.copy(helloBuffer, helloStringLength, 0);

console.log(helloBuffer.toString('utf-8'));

Cropping the buffer

buf.slice([start], [end]) method returns a new buffer which references the same memory as the old, but offset and cropped by the start (defaults to 0) and end (defaults to buffer.length) indexes. The slice is not a new buffer and merely references a subset of the memory space. Modifying the slice will also modify the original buffer. For example:

var nodejsSlice = helloBuffer.slice(helloStringLength, helloStringLength + welcomeStringLength);

console.log(nodejsSlice.toString());

nodejsSlice.write('Prajeesh');

console.log(helloBuffer.toString());